9cdca4137d
* [forwardport] fix(oonimkall): make logger used by tasks unit testable (#623) This diff forward ports e4b04642c51e7461728b25941624e1b97ef0ec83. Reference issue: https://github.com/ooni/probe/issues/1903 * [forwardport] feat(oonimkall): improve taskEmitter testability (#624) This diff forward ports 3e0f01a389c1f4cdd7878ec151aff91870a0bdff. 1. rename eventemitter{,_test}.go => taskemitter{,_test}.go because the new name is more proper after we merged the internal/task package inside of the oonimkall package; 2. rename runner.go's `run` function to `runTask`; 3. modify `runTask` to use the new `taskEmitterUsingChan` abstraction on which we will spend more works in a later point of this list; 4. introduce `runTaskWithEmitter` factory that is called by `runTask` and allows us to more easily write unit tests; 5. acknowledge that `runner` was not using its `out` field; 6. use the new `taskEmitterWrapper` in `newRunner`; 7. acknowledge that `runnerCallbacks` could use a generic `taskEmitter` as field type rather than a specific type; 8. rewrite tests to use `runTaskWithEmitter` which leads to simpler code that does not require a goroutine; 9. acknowledge that the code has been ignoring the `DisabledEvents` settings for quite some time, so stop supporting it; 10. refactor the `taskEmitter` implementation to be like: 1. we still have the `taskEmitter` interface; 2. `taskEmitterUsingChan` wraps the channel and allows for emitting events using the channel; 3. `taskEmitterUsingChan` owns an `eof` channel that is closed by `Close` (which is idempotent) and signals we should be stop emitting; 4. make sure `runTask` creates a `taskEmitterUsingChan` and calls its `Close` method when done; 5. completely remove the code for disabling events since the code was actually ignoring the stting; 6. add a `taskEmitterWrapper` that adds common functions for emitting events to _any_ `taskWrapper`; 7. write unit tests for `taskEmitterUsingChan` and for `taskEmitterWrapper`; 11. acknowledge that the abstraction we need for testing is actually a thread-safe thing that collects events into a vector containing events and refactor all tests accordingly. See https://github.com/ooni/probe/issues/1903 * [forwardport] refactor(oonimkall): make the runner unit-testable (#625) This diff forward ports 9423947faf6980d92d2fe67efe3829e8fef76586. See https://github.com/ooni/probe/issues/1903 * [forwardport] feat(oonimkall): write unit tests for the runner component (#626) This diff forward ports 35dd0e3788b8fa99c541452bbb5e0ae4871239e1. Forward porting note: compared to 35dd0e3788b8fa99c541452bbb5e0ae4871239e1, the diff I'm committing here is slightly different. In `master` we do not have the case where a measurement fails and a measurement is returned, thus I needed to adapt the test to become like this: ```diff diff --git a/pkg/oonimkall/runner_internal_test.go b/pkg/oonimkall/runner_internal_test.go index 334b574..84c7436 100644 --- a/pkg/oonimkall/runner_internal_test.go +++ b/pkg/oonimkall/runner_internal_test.go @@ -568,15 +568,6 @@ func TestTaskRunnerRun(t *testing.T) { }, { Key: failureMeasurement, Count: 1, - }, { - Key: measurement, - Count: 1, - }, { - Key: statusMeasurementSubmission, - Count: 1, - }, { - Key: statusMeasurementDone, - Count: 1, }, { Key: statusEnd, Count: 1, ``` I still need to write more assertions for each emitted event but the code we've here is already a great starting point. See https://github.com/ooni/probe/issues/1903 * [forwardport] refactor(oonimkall): merge files, use proper names, zap unneeded integration tests (#627) This diff forward ports f894427d24edc9a03fc78306d0093e7b51c46c25. Forward porting note: this diff is slightly different from the original mentioned above because it carries forward changes mentioned in the previous diff caused by a different way of handling a failed measurement in the master branch compared to the release/3.11 branch. Move everything that looked like "task's model" inside of the taskmodel.go file, for consistency. Make sure it's clear some variables are event types. Rename the concrete `runner` as `runnerForTask`. Also, remove now-unnecessary (and flaky!) integration tests for the `runnerForTask` type. While there, notice there were wrong URLs that were generated during the probe-engine => probe-cli move and fix them. See https://github.com/ooni/probe/issues/1903 * [forwardport] refactor(oonimkall): we can simplify StartTask tests (#628) This diff forward ports dcf2986c2032d8185d58d24130a7f2c2d61ef2fb. * refactor(oonimkall): we can simplify StartTask tests We have enough checks for runnerForTask. So we do not need to duplicate them when checking for StartTask. While there, refactor how we start tasks to remove the need for extra runner functions. This is the objective I wanted to achieve for oonimkall: 1. less duplicate tests, and 2. more unit tests (which are less flaky) At this point, we're basically done (pending forwardporting to master) with https://github.com/ooni/probe/issues/1903. * fix(oonimkall): TestStartTaskGood shouldn't cancel the test This creates a race condition where the test may fail if we cannot complete the whole "Example" test in less than one second. This should explain the build failures I've seen so far and why I didn't see those failures when running locally.
123 lines
4.1 KiB
Go
123 lines
4.1 KiB
Go
// Package oonimkall implements APIs used by OONI mobile apps. We
|
|
// expose these APIs to mobile apps using gomobile.
|
|
//
|
|
// We expose two APIs: the task API, which is derived from the
|
|
// API originally exposed by Measurement Kit, and the session API,
|
|
// which is a Go API that mobile apps can use via `gomobile`.
|
|
//
|
|
// This package is named oonimkall because it contains a partial
|
|
// reimplementation of the mkall API implemented by Measurement Kit
|
|
// in, e.g., https://github.com/measurement-kit/mkall-ios.
|
|
//
|
|
// Task API
|
|
//
|
|
// The basic tenet of the task API is that you define an experiment
|
|
// task you wanna run using a JSON, then you start a task for it, and
|
|
// you receive events as serialized JSONs. In addition to this
|
|
// functionality, we also include extra APIs used by OONI mobile.
|
|
//
|
|
// The task API was first defined in Measurement Kit v0.9.0. In this
|
|
// context, it was called "the FFI API". The API we expose here is not
|
|
// strictly an FFI API, but is close enough for the purpose of using
|
|
// OONI from Android and iOS. See https://git.io/Jv4Rv
|
|
// (measurement-kit/measurement-kit@v0.10.9) for a comprehensive
|
|
// description of MK's FFI API.
|
|
//
|
|
// See also https://github.com/ooni/probe-engine/pull/347 for the
|
|
// design document describing the task API.
|
|
//
|
|
// See also https://github.com/ooni/probe-cli/v3/internal/engine/blob/master/DESIGN.md,
|
|
// which explains why we implemented the oonimkall API.
|
|
//
|
|
// Session API
|
|
//
|
|
// The Session API is a Go API that can be exported to mobile apps
|
|
// using the gomobile tool. The latest design document for this API is
|
|
// at https://github.com/ooni/probe-engine/pull/954.
|
|
//
|
|
// The basic tenet of the session API is that you create an instance
|
|
// of `Session` and use it to perform the operations you need.
|
|
package oonimkall
|
|
|
|
import (
|
|
"context"
|
|
"encoding/json"
|
|
|
|
"github.com/ooni/probe-cli/v3/internal/atomicx"
|
|
"github.com/ooni/probe-cli/v3/internal/runtimex"
|
|
)
|
|
|
|
// Task is an asynchronous task running an experiment. It mimics the
|
|
// namesake concept initially implemented in Measurement Kit.
|
|
//
|
|
// Future directions
|
|
//
|
|
// Currently Task and Session are two unrelated APIs. As part of
|
|
// evolving the APIs with which apps interact with the engine, we
|
|
// will modify Task to run in the context of a Session. We will
|
|
// do that to save extra lookups and to allow several experiments
|
|
// running as subsequent Tasks to reuse the Session connections
|
|
// created with the OONI probe services backends.
|
|
type Task struct {
|
|
cancel context.CancelFunc
|
|
isdone *atomicx.Int64
|
|
isstarted chan interface{} // for testing
|
|
isstopped chan interface{} // for testing
|
|
out chan *event
|
|
}
|
|
|
|
// StartTask starts an asynchronous task. The input argument is a
|
|
// serialized JSON conforming to MK v0.10.9's API.
|
|
func StartTask(input string) (*Task, error) {
|
|
var settings settings
|
|
if err := json.Unmarshal([]byte(input), &settings); err != nil {
|
|
return nil, err
|
|
}
|
|
const bufsiz = 128 // common case: we don't want runner to block
|
|
ctx, cancel := context.WithCancel(context.Background())
|
|
task := &Task{
|
|
cancel: cancel,
|
|
isdone: &atomicx.Int64{},
|
|
isstarted: make(chan interface{}),
|
|
isstopped: make(chan interface{}),
|
|
out: make(chan *event, bufsiz),
|
|
}
|
|
go func() {
|
|
close(task.isstarted)
|
|
emitter := newTaskEmitterUsingChan(task.out)
|
|
r := newRunner(&settings, emitter)
|
|
r.Run(ctx)
|
|
task.out <- nil // signal that we're done w/o closing the channel
|
|
emitter.Close()
|
|
close(task.isstopped)
|
|
}()
|
|
return task, nil
|
|
}
|
|
|
|
// WaitForNextEvent blocks until the next event occurs. The returned
|
|
// string is a serialized JSON following MK v0.10.9's API.
|
|
func (t *Task) WaitForNextEvent() string {
|
|
const terminated = `{"key":"task_terminated","value":{}}` // like MK
|
|
if t.isdone.Load() != 0 {
|
|
return terminated
|
|
}
|
|
evp := <-t.out
|
|
if evp == nil {
|
|
t.isdone.Add(1)
|
|
return terminated
|
|
}
|
|
data, err := json.Marshal(evp)
|
|
runtimex.PanicOnError(err, "json.Marshal failed")
|
|
return string(data)
|
|
}
|
|
|
|
// IsDone returns true if the task is done.
|
|
func (t *Task) IsDone() bool {
|
|
return t.isdone.Load() != 0
|
|
}
|
|
|
|
// Interrupt interrupts the task.
|
|
func (t *Task) Interrupt() {
|
|
t.cancel()
|
|
}
|