forwardport: pull the patches mentioned in ooni/probe#1908 (#629)
* [forwardport] fix(oonimkall): make logger used by tasks unit testable (#623)
This diff forward ports e4b04642c51e7461728b25941624e1b97ef0ec83.
Reference issue: https://github.com/ooni/probe/issues/1903
* [forwardport] feat(oonimkall): improve taskEmitter testability (#624)
This diff forward ports 3e0f01a389c1f4cdd7878ec151aff91870a0bdff.
1. rename eventemitter{,_test}.go => taskemitter{,_test}.go because
the new name is more proper after we merged the internal/task package
inside of the oonimkall package;
2. rename runner.go's `run` function to `runTask`;
3. modify `runTask` to use the new `taskEmitterUsingChan` abstraction
on which we will spend more works in a later point of this list;
4. introduce `runTaskWithEmitter` factory that is called by `runTask`
and allows us to more easily write unit tests;
5. acknowledge that `runner` was not using its `out` field;
6. use the new `taskEmitterWrapper` in `newRunner`;
7. acknowledge that `runnerCallbacks` could use a generic
`taskEmitter` as field type rather than a specific type;
8. rewrite tests to use `runTaskWithEmitter` which leads to
simpler code that does not require a goroutine;
9. acknowledge that the code has been ignoring the `DisabledEvents`
settings for quite some time, so stop supporting it;
10. refactor the `taskEmitter` implementation to be like:
1. we still have the `taskEmitter` interface;
2. `taskEmitterUsingChan` wraps the channel and allows for
emitting events using the channel;
3. `taskEmitterUsingChan` owns an `eof` channel that is
closed by `Close` (which is idempotent) and signals we
should be stop emitting;
4. make sure `runTask` creates a `taskEmitterUsingChan`
and calls its `Close` method when done;
5. completely remove the code for disabling events
since the code was actually ignoring the stting;
6. add a `taskEmitterWrapper` that adds common functions
for emitting events to _any_ `taskWrapper`;
7. write unit tests for `taskEmitterUsingChan` and
for `taskEmitterWrapper`;
11. acknowledge that the abstraction we need for testing is
actually a thread-safe thing that collects events into a
vector containing events and refactor all tests accordingly.
See https://github.com/ooni/probe/issues/1903
* [forwardport] refactor(oonimkall): make the runner unit-testable (#625)
This diff forward ports 9423947faf6980d92d2fe67efe3829e8fef76586.
See https://github.com/ooni/probe/issues/1903
* [forwardport] feat(oonimkall): write unit tests for the runner component (#626)
This diff forward ports 35dd0e3788b8fa99c541452bbb5e0ae4871239e1.
Forward porting note: compared to 35dd0e3788b8fa99c541452bbb5e0ae4871239e1,
the diff I'm committing here is slightly different. In `master` we do not
have the case where a measurement fails and a measurement is returned, thus
I needed to adapt the test to become like this:
```diff
diff --git a/pkg/oonimkall/runner_internal_test.go b/pkg/oonimkall/runner_internal_test.go
index 334b574..84c7436 100644
--- a/pkg/oonimkall/runner_internal_test.go
+++ b/pkg/oonimkall/runner_internal_test.go
@@ -568,15 +568,6 @@ func TestTaskRunnerRun(t *testing.T) {
}, {
Key: failureMeasurement,
Count: 1,
- }, {
- Key: measurement,
- Count: 1,
- }, {
- Key: statusMeasurementSubmission,
- Count: 1,
- }, {
- Key: statusMeasurementDone,
- Count: 1,
}, {
Key: statusEnd,
Count: 1,
```
I still need to write more assertions for each emitted event
but the code we've here is already a great starting point.
See https://github.com/ooni/probe/issues/1903
* [forwardport] refactor(oonimkall): merge files, use proper names, zap unneeded integration tests (#627)
This diff forward ports f894427d24edc9a03fc78306d0093e7b51c46c25.
Forward porting note: this diff is slightly different from the original
mentioned above because it carries forward changes mentioned in the
previous diff caused by a different way of handling a failed measurement
in the master branch compared to the release/3.11 branch.
Move everything that looked like "task's model" inside of the
taskmodel.go file, for consistency.
Make sure it's clear some variables are event types.
Rename the concrete `runner` as `runnerForTask`.
Also, remove now-unnecessary (and flaky!) integration tests
for the `runnerForTask` type.
While there, notice there were wrong URLs that were generated
during the probe-engine => probe-cli move and fix them.
See https://github.com/ooni/probe/issues/1903
* [forwardport] refactor(oonimkall): we can simplify StartTask tests (#628)
This diff forward ports dcf2986c2032d8185d58d24130a7f2c2d61ef2fb.
* refactor(oonimkall): we can simplify StartTask tests
We have enough checks for runnerForTask. So we do not need to
duplicate them when checking for StartTask.
While there, refactor how we start tasks to remove the need for
extra runner functions.
This is the objective I wanted to achieve for oonimkall:
1. less duplicate tests, and
2. more unit tests (which are less flaky)
At this point, we're basically done (pending forwardporting to
master) with https://github.com/ooni/probe/issues/1903.
* fix(oonimkall): TestStartTaskGood shouldn't cancel the test
This creates a race condition where the test may fail if we cannot
complete the whole "Example" test in less than one second.
This should explain the build failures I've seen so far and why
I didn't see those failures when running locally.
2021-12-02 12:47:07 +01:00
|
|
|
package oonimkall
|
|
|
|
|
|
|
|
import (
|
|
|
|
"context"
|
|
|
|
"errors"
|
|
|
|
"sync"
|
|
|
|
|
|
|
|
"github.com/ooni/probe-cli/v3/internal/engine"
|
2022-01-03 13:53:23 +01:00
|
|
|
"github.com/ooni/probe-cli/v3/internal/model"
|
forwardport: pull the patches mentioned in ooni/probe#1908 (#629)
* [forwardport] fix(oonimkall): make logger used by tasks unit testable (#623)
This diff forward ports e4b04642c51e7461728b25941624e1b97ef0ec83.
Reference issue: https://github.com/ooni/probe/issues/1903
* [forwardport] feat(oonimkall): improve taskEmitter testability (#624)
This diff forward ports 3e0f01a389c1f4cdd7878ec151aff91870a0bdff.
1. rename eventemitter{,_test}.go => taskemitter{,_test}.go because
the new name is more proper after we merged the internal/task package
inside of the oonimkall package;
2. rename runner.go's `run` function to `runTask`;
3. modify `runTask` to use the new `taskEmitterUsingChan` abstraction
on which we will spend more works in a later point of this list;
4. introduce `runTaskWithEmitter` factory that is called by `runTask`
and allows us to more easily write unit tests;
5. acknowledge that `runner` was not using its `out` field;
6. use the new `taskEmitterWrapper` in `newRunner`;
7. acknowledge that `runnerCallbacks` could use a generic
`taskEmitter` as field type rather than a specific type;
8. rewrite tests to use `runTaskWithEmitter` which leads to
simpler code that does not require a goroutine;
9. acknowledge that the code has been ignoring the `DisabledEvents`
settings for quite some time, so stop supporting it;
10. refactor the `taskEmitter` implementation to be like:
1. we still have the `taskEmitter` interface;
2. `taskEmitterUsingChan` wraps the channel and allows for
emitting events using the channel;
3. `taskEmitterUsingChan` owns an `eof` channel that is
closed by `Close` (which is idempotent) and signals we
should be stop emitting;
4. make sure `runTask` creates a `taskEmitterUsingChan`
and calls its `Close` method when done;
5. completely remove the code for disabling events
since the code was actually ignoring the stting;
6. add a `taskEmitterWrapper` that adds common functions
for emitting events to _any_ `taskWrapper`;
7. write unit tests for `taskEmitterUsingChan` and
for `taskEmitterWrapper`;
11. acknowledge that the abstraction we need for testing is
actually a thread-safe thing that collects events into a
vector containing events and refactor all tests accordingly.
See https://github.com/ooni/probe/issues/1903
* [forwardport] refactor(oonimkall): make the runner unit-testable (#625)
This diff forward ports 9423947faf6980d92d2fe67efe3829e8fef76586.
See https://github.com/ooni/probe/issues/1903
* [forwardport] feat(oonimkall): write unit tests for the runner component (#626)
This diff forward ports 35dd0e3788b8fa99c541452bbb5e0ae4871239e1.
Forward porting note: compared to 35dd0e3788b8fa99c541452bbb5e0ae4871239e1,
the diff I'm committing here is slightly different. In `master` we do not
have the case where a measurement fails and a measurement is returned, thus
I needed to adapt the test to become like this:
```diff
diff --git a/pkg/oonimkall/runner_internal_test.go b/pkg/oonimkall/runner_internal_test.go
index 334b574..84c7436 100644
--- a/pkg/oonimkall/runner_internal_test.go
+++ b/pkg/oonimkall/runner_internal_test.go
@@ -568,15 +568,6 @@ func TestTaskRunnerRun(t *testing.T) {
}, {
Key: failureMeasurement,
Count: 1,
- }, {
- Key: measurement,
- Count: 1,
- }, {
- Key: statusMeasurementSubmission,
- Count: 1,
- }, {
- Key: statusMeasurementDone,
- Count: 1,
}, {
Key: statusEnd,
Count: 1,
```
I still need to write more assertions for each emitted event
but the code we've here is already a great starting point.
See https://github.com/ooni/probe/issues/1903
* [forwardport] refactor(oonimkall): merge files, use proper names, zap unneeded integration tests (#627)
This diff forward ports f894427d24edc9a03fc78306d0093e7b51c46c25.
Forward porting note: this diff is slightly different from the original
mentioned above because it carries forward changes mentioned in the
previous diff caused by a different way of handling a failed measurement
in the master branch compared to the release/3.11 branch.
Move everything that looked like "task's model" inside of the
taskmodel.go file, for consistency.
Make sure it's clear some variables are event types.
Rename the concrete `runner` as `runnerForTask`.
Also, remove now-unnecessary (and flaky!) integration tests
for the `runnerForTask` type.
While there, notice there were wrong URLs that were generated
during the probe-engine => probe-cli move and fix them.
See https://github.com/ooni/probe/issues/1903
* [forwardport] refactor(oonimkall): we can simplify StartTask tests (#628)
This diff forward ports dcf2986c2032d8185d58d24130a7f2c2d61ef2fb.
* refactor(oonimkall): we can simplify StartTask tests
We have enough checks for runnerForTask. So we do not need to
duplicate them when checking for StartTask.
While there, refactor how we start tasks to remove the need for
extra runner functions.
This is the objective I wanted to achieve for oonimkall:
1. less duplicate tests, and
2. more unit tests (which are less flaky)
At this point, we're basically done (pending forwardporting to
master) with https://github.com/ooni/probe/issues/1903.
* fix(oonimkall): TestStartTaskGood shouldn't cancel the test
This creates a race condition where the test may fail if we cannot
complete the whole "Example" test in less than one second.
This should explain the build failures I've seen so far and why
I didn't see those failures when running locally.
2021-12-02 12:47:07 +01:00
|
|
|
)
|
|
|
|
|
|
|
|
//
|
|
|
|
// This file contains mocks for types used by tasks. Because
|
|
|
|
// we only use mocks when testing, this file is a `_test.go` file.
|
|
|
|
//
|
|
|
|
|
|
|
|
// CollectorTaskEmitter is a thread-safe taskEmitter
|
|
|
|
// that stores all the events inside itself.
|
|
|
|
type CollectorTaskEmitter struct {
|
|
|
|
// events contains the events
|
|
|
|
events []*event
|
|
|
|
|
|
|
|
// mu provides mutual exclusion
|
|
|
|
mu sync.Mutex
|
|
|
|
}
|
|
|
|
|
|
|
|
// ensures that a CollectorTaskEmitter is a taskEmitter.
|
|
|
|
var _ taskEmitter = &CollectorTaskEmitter{}
|
|
|
|
|
|
|
|
// Emit implements the taskEmitter.Emit method.
|
|
|
|
func (e *CollectorTaskEmitter) Emit(key string, value interface{}) {
|
|
|
|
e.mu.Lock()
|
|
|
|
e.events = append(e.events, &event{Key: key, Value: value})
|
|
|
|
e.mu.Unlock()
|
|
|
|
}
|
|
|
|
|
|
|
|
// Collect returns a copy of the collected events. It is safe
|
|
|
|
// to read the events. It's a data race to modify them.
|
|
|
|
//
|
|
|
|
// After this function has been called, the internal array
|
|
|
|
// of events will now be empty.
|
|
|
|
func (e *CollectorTaskEmitter) Collect() (out []*event) {
|
|
|
|
e.mu.Lock()
|
|
|
|
out = e.events
|
|
|
|
e.events = nil
|
|
|
|
e.mu.Unlock()
|
|
|
|
return
|
|
|
|
}
|
|
|
|
|
|
|
|
// SessionBuilderConfigSaver is a session builder that
|
|
|
|
// saves the received config and returns an error.
|
|
|
|
type SessionBuilderConfigSaver struct {
|
|
|
|
Config engine.SessionConfig
|
|
|
|
}
|
|
|
|
|
|
|
|
var _ taskSessionBuilder = &SessionBuilderConfigSaver{}
|
|
|
|
|
|
|
|
func (b *SessionBuilderConfigSaver) NewSession(
|
|
|
|
ctx context.Context, config engine.SessionConfig) (taskSession, error) {
|
|
|
|
b.Config = config
|
|
|
|
return nil, errors.New("mocked error")
|
|
|
|
}
|
|
|
|
|
|
|
|
// MockableTaskRunnerDependencies allows to mock all the
|
|
|
|
// dependencies of taskRunner using a single structure.
|
|
|
|
type MockableTaskRunnerDependencies struct {
|
|
|
|
|
|
|
|
// taskSessionBuilder:
|
|
|
|
|
|
|
|
MockNewSession func(ctx context.Context,
|
|
|
|
config engine.SessionConfig) (taskSession, error)
|
|
|
|
|
|
|
|
// taskSession:
|
|
|
|
|
|
|
|
MockClose func() error
|
|
|
|
MockNewExperimentBuilderByName func(name string) (taskExperimentBuilder, error)
|
|
|
|
MockMaybeLookupBackendsContext func(ctx context.Context) error
|
|
|
|
MockMaybeLookupLocationContext func(ctx context.Context) error
|
|
|
|
MockProbeIP func() string
|
|
|
|
MockProbeASNString func() string
|
|
|
|
MockProbeCC func() string
|
|
|
|
MockProbeNetworkName func() string
|
|
|
|
MockResolverASNString func() string
|
|
|
|
MockResolverIP func() string
|
|
|
|
MockResolverNetworkName func() string
|
|
|
|
|
|
|
|
// taskExperimentBuilder:
|
|
|
|
|
|
|
|
MockableSetCallbacks func(callbacks model.ExperimentCallbacks)
|
2022-08-17 10:57:03 +02:00
|
|
|
MockableInputPolicy func() model.InputPolicy
|
forwardport: pull the patches mentioned in ooni/probe#1908 (#629)
* [forwardport] fix(oonimkall): make logger used by tasks unit testable (#623)
This diff forward ports e4b04642c51e7461728b25941624e1b97ef0ec83.
Reference issue: https://github.com/ooni/probe/issues/1903
* [forwardport] feat(oonimkall): improve taskEmitter testability (#624)
This diff forward ports 3e0f01a389c1f4cdd7878ec151aff91870a0bdff.
1. rename eventemitter{,_test}.go => taskemitter{,_test}.go because
the new name is more proper after we merged the internal/task package
inside of the oonimkall package;
2. rename runner.go's `run` function to `runTask`;
3. modify `runTask` to use the new `taskEmitterUsingChan` abstraction
on which we will spend more works in a later point of this list;
4. introduce `runTaskWithEmitter` factory that is called by `runTask`
and allows us to more easily write unit tests;
5. acknowledge that `runner` was not using its `out` field;
6. use the new `taskEmitterWrapper` in `newRunner`;
7. acknowledge that `runnerCallbacks` could use a generic
`taskEmitter` as field type rather than a specific type;
8. rewrite tests to use `runTaskWithEmitter` which leads to
simpler code that does not require a goroutine;
9. acknowledge that the code has been ignoring the `DisabledEvents`
settings for quite some time, so stop supporting it;
10. refactor the `taskEmitter` implementation to be like:
1. we still have the `taskEmitter` interface;
2. `taskEmitterUsingChan` wraps the channel and allows for
emitting events using the channel;
3. `taskEmitterUsingChan` owns an `eof` channel that is
closed by `Close` (which is idempotent) and signals we
should be stop emitting;
4. make sure `runTask` creates a `taskEmitterUsingChan`
and calls its `Close` method when done;
5. completely remove the code for disabling events
since the code was actually ignoring the stting;
6. add a `taskEmitterWrapper` that adds common functions
for emitting events to _any_ `taskWrapper`;
7. write unit tests for `taskEmitterUsingChan` and
for `taskEmitterWrapper`;
11. acknowledge that the abstraction we need for testing is
actually a thread-safe thing that collects events into a
vector containing events and refactor all tests accordingly.
See https://github.com/ooni/probe/issues/1903
* [forwardport] refactor(oonimkall): make the runner unit-testable (#625)
This diff forward ports 9423947faf6980d92d2fe67efe3829e8fef76586.
See https://github.com/ooni/probe/issues/1903
* [forwardport] feat(oonimkall): write unit tests for the runner component (#626)
This diff forward ports 35dd0e3788b8fa99c541452bbb5e0ae4871239e1.
Forward porting note: compared to 35dd0e3788b8fa99c541452bbb5e0ae4871239e1,
the diff I'm committing here is slightly different. In `master` we do not
have the case where a measurement fails and a measurement is returned, thus
I needed to adapt the test to become like this:
```diff
diff --git a/pkg/oonimkall/runner_internal_test.go b/pkg/oonimkall/runner_internal_test.go
index 334b574..84c7436 100644
--- a/pkg/oonimkall/runner_internal_test.go
+++ b/pkg/oonimkall/runner_internal_test.go
@@ -568,15 +568,6 @@ func TestTaskRunnerRun(t *testing.T) {
}, {
Key: failureMeasurement,
Count: 1,
- }, {
- Key: measurement,
- Count: 1,
- }, {
- Key: statusMeasurementSubmission,
- Count: 1,
- }, {
- Key: statusMeasurementDone,
- Count: 1,
}, {
Key: statusEnd,
Count: 1,
```
I still need to write more assertions for each emitted event
but the code we've here is already a great starting point.
See https://github.com/ooni/probe/issues/1903
* [forwardport] refactor(oonimkall): merge files, use proper names, zap unneeded integration tests (#627)
This diff forward ports f894427d24edc9a03fc78306d0093e7b51c46c25.
Forward porting note: this diff is slightly different from the original
mentioned above because it carries forward changes mentioned in the
previous diff caused by a different way of handling a failed measurement
in the master branch compared to the release/3.11 branch.
Move everything that looked like "task's model" inside of the
taskmodel.go file, for consistency.
Make sure it's clear some variables are event types.
Rename the concrete `runner` as `runnerForTask`.
Also, remove now-unnecessary (and flaky!) integration tests
for the `runnerForTask` type.
While there, notice there were wrong URLs that were generated
during the probe-engine => probe-cli move and fix them.
See https://github.com/ooni/probe/issues/1903
* [forwardport] refactor(oonimkall): we can simplify StartTask tests (#628)
This diff forward ports dcf2986c2032d8185d58d24130a7f2c2d61ef2fb.
* refactor(oonimkall): we can simplify StartTask tests
We have enough checks for runnerForTask. So we do not need to
duplicate them when checking for StartTask.
While there, refactor how we start tasks to remove the need for
extra runner functions.
This is the objective I wanted to achieve for oonimkall:
1. less duplicate tests, and
2. more unit tests (which are less flaky)
At this point, we're basically done (pending forwardporting to
master) with https://github.com/ooni/probe/issues/1903.
* fix(oonimkall): TestStartTaskGood shouldn't cancel the test
This creates a race condition where the test may fail if we cannot
complete the whole "Example" test in less than one second.
This should explain the build failures I've seen so far and why
I didn't see those failures when running locally.
2021-12-02 12:47:07 +01:00
|
|
|
MockableNewExperimentInstance func() taskExperiment
|
|
|
|
MockableInterruptible func() bool
|
|
|
|
|
|
|
|
// taskExperiment:
|
|
|
|
|
|
|
|
MockableKibiBytesReceived func() float64
|
|
|
|
MockableKibiBytesSent func() float64
|
|
|
|
MockableOpenReportContext func(ctx context.Context) error
|
|
|
|
MockableReportID func() string
|
|
|
|
MockableMeasureWithContext func(ctx context.Context, input string) (
|
|
|
|
measurement *model.Measurement, err error)
|
|
|
|
MockableSubmitAndUpdateMeasurementContext func(
|
|
|
|
ctx context.Context, measurement *model.Measurement) error
|
|
|
|
}
|
|
|
|
|
|
|
|
var (
|
|
|
|
_ taskSessionBuilder = &MockableTaskRunnerDependencies{}
|
|
|
|
_ taskSession = &MockableTaskRunnerDependencies{}
|
|
|
|
_ taskExperimentBuilder = &MockableTaskRunnerDependencies{}
|
|
|
|
_ taskExperiment = &MockableTaskRunnerDependencies{}
|
|
|
|
)
|
|
|
|
|
|
|
|
func (dep *MockableTaskRunnerDependencies) NewSession(
|
|
|
|
ctx context.Context, config engine.SessionConfig) (taskSession, error) {
|
|
|
|
if f := dep.MockNewSession; f != nil {
|
|
|
|
return f(ctx, config)
|
|
|
|
}
|
|
|
|
return dep, nil
|
|
|
|
}
|
|
|
|
|
|
|
|
func (dep *MockableTaskRunnerDependencies) Close() error {
|
|
|
|
return dep.MockClose()
|
|
|
|
}
|
|
|
|
|
|
|
|
func (dep *MockableTaskRunnerDependencies) NewExperimentBuilderByName(name string) (taskExperimentBuilder, error) {
|
|
|
|
if f := dep.MockNewExperimentBuilderByName; f != nil {
|
|
|
|
return f(name)
|
|
|
|
}
|
|
|
|
return dep, nil
|
|
|
|
}
|
|
|
|
|
|
|
|
func (dep *MockableTaskRunnerDependencies) MaybeLookupBackendsContext(ctx context.Context) error {
|
|
|
|
return dep.MockMaybeLookupBackendsContext(ctx)
|
|
|
|
}
|
|
|
|
|
|
|
|
func (dep *MockableTaskRunnerDependencies) MaybeLookupLocationContext(ctx context.Context) error {
|
|
|
|
return dep.MockMaybeLookupLocationContext(ctx)
|
|
|
|
}
|
|
|
|
|
|
|
|
func (dep *MockableTaskRunnerDependencies) ProbeIP() string {
|
|
|
|
return dep.MockProbeIP()
|
|
|
|
}
|
|
|
|
|
|
|
|
func (dep *MockableTaskRunnerDependencies) ProbeASNString() string {
|
|
|
|
return dep.MockProbeASNString()
|
|
|
|
}
|
|
|
|
|
|
|
|
func (dep *MockableTaskRunnerDependencies) ProbeCC() string {
|
|
|
|
return dep.MockProbeCC()
|
|
|
|
}
|
|
|
|
|
|
|
|
func (dep *MockableTaskRunnerDependencies) ProbeNetworkName() string {
|
|
|
|
return dep.MockProbeNetworkName()
|
|
|
|
}
|
|
|
|
|
|
|
|
func (dep *MockableTaskRunnerDependencies) ResolverASNString() string {
|
|
|
|
return dep.MockResolverASNString()
|
|
|
|
}
|
|
|
|
|
|
|
|
func (dep *MockableTaskRunnerDependencies) ResolverIP() string {
|
|
|
|
return dep.MockResolverIP()
|
|
|
|
}
|
|
|
|
|
|
|
|
func (dep *MockableTaskRunnerDependencies) ResolverNetworkName() string {
|
|
|
|
return dep.MockResolverNetworkName()
|
|
|
|
}
|
|
|
|
|
|
|
|
func (dep *MockableTaskRunnerDependencies) SetCallbacks(callbacks model.ExperimentCallbacks) {
|
|
|
|
dep.MockableSetCallbacks(callbacks)
|
|
|
|
}
|
|
|
|
|
2022-08-17 10:57:03 +02:00
|
|
|
func (dep *MockableTaskRunnerDependencies) InputPolicy() model.InputPolicy {
|
forwardport: pull the patches mentioned in ooni/probe#1908 (#629)
* [forwardport] fix(oonimkall): make logger used by tasks unit testable (#623)
This diff forward ports e4b04642c51e7461728b25941624e1b97ef0ec83.
Reference issue: https://github.com/ooni/probe/issues/1903
* [forwardport] feat(oonimkall): improve taskEmitter testability (#624)
This diff forward ports 3e0f01a389c1f4cdd7878ec151aff91870a0bdff.
1. rename eventemitter{,_test}.go => taskemitter{,_test}.go because
the new name is more proper after we merged the internal/task package
inside of the oonimkall package;
2. rename runner.go's `run` function to `runTask`;
3. modify `runTask` to use the new `taskEmitterUsingChan` abstraction
on which we will spend more works in a later point of this list;
4. introduce `runTaskWithEmitter` factory that is called by `runTask`
and allows us to more easily write unit tests;
5. acknowledge that `runner` was not using its `out` field;
6. use the new `taskEmitterWrapper` in `newRunner`;
7. acknowledge that `runnerCallbacks` could use a generic
`taskEmitter` as field type rather than a specific type;
8. rewrite tests to use `runTaskWithEmitter` which leads to
simpler code that does not require a goroutine;
9. acknowledge that the code has been ignoring the `DisabledEvents`
settings for quite some time, so stop supporting it;
10. refactor the `taskEmitter` implementation to be like:
1. we still have the `taskEmitter` interface;
2. `taskEmitterUsingChan` wraps the channel and allows for
emitting events using the channel;
3. `taskEmitterUsingChan` owns an `eof` channel that is
closed by `Close` (which is idempotent) and signals we
should be stop emitting;
4. make sure `runTask` creates a `taskEmitterUsingChan`
and calls its `Close` method when done;
5. completely remove the code for disabling events
since the code was actually ignoring the stting;
6. add a `taskEmitterWrapper` that adds common functions
for emitting events to _any_ `taskWrapper`;
7. write unit tests for `taskEmitterUsingChan` and
for `taskEmitterWrapper`;
11. acknowledge that the abstraction we need for testing is
actually a thread-safe thing that collects events into a
vector containing events and refactor all tests accordingly.
See https://github.com/ooni/probe/issues/1903
* [forwardport] refactor(oonimkall): make the runner unit-testable (#625)
This diff forward ports 9423947faf6980d92d2fe67efe3829e8fef76586.
See https://github.com/ooni/probe/issues/1903
* [forwardport] feat(oonimkall): write unit tests for the runner component (#626)
This diff forward ports 35dd0e3788b8fa99c541452bbb5e0ae4871239e1.
Forward porting note: compared to 35dd0e3788b8fa99c541452bbb5e0ae4871239e1,
the diff I'm committing here is slightly different. In `master` we do not
have the case where a measurement fails and a measurement is returned, thus
I needed to adapt the test to become like this:
```diff
diff --git a/pkg/oonimkall/runner_internal_test.go b/pkg/oonimkall/runner_internal_test.go
index 334b574..84c7436 100644
--- a/pkg/oonimkall/runner_internal_test.go
+++ b/pkg/oonimkall/runner_internal_test.go
@@ -568,15 +568,6 @@ func TestTaskRunnerRun(t *testing.T) {
}, {
Key: failureMeasurement,
Count: 1,
- }, {
- Key: measurement,
- Count: 1,
- }, {
- Key: statusMeasurementSubmission,
- Count: 1,
- }, {
- Key: statusMeasurementDone,
- Count: 1,
}, {
Key: statusEnd,
Count: 1,
```
I still need to write more assertions for each emitted event
but the code we've here is already a great starting point.
See https://github.com/ooni/probe/issues/1903
* [forwardport] refactor(oonimkall): merge files, use proper names, zap unneeded integration tests (#627)
This diff forward ports f894427d24edc9a03fc78306d0093e7b51c46c25.
Forward porting note: this diff is slightly different from the original
mentioned above because it carries forward changes mentioned in the
previous diff caused by a different way of handling a failed measurement
in the master branch compared to the release/3.11 branch.
Move everything that looked like "task's model" inside of the
taskmodel.go file, for consistency.
Make sure it's clear some variables are event types.
Rename the concrete `runner` as `runnerForTask`.
Also, remove now-unnecessary (and flaky!) integration tests
for the `runnerForTask` type.
While there, notice there were wrong URLs that were generated
during the probe-engine => probe-cli move and fix them.
See https://github.com/ooni/probe/issues/1903
* [forwardport] refactor(oonimkall): we can simplify StartTask tests (#628)
This diff forward ports dcf2986c2032d8185d58d24130a7f2c2d61ef2fb.
* refactor(oonimkall): we can simplify StartTask tests
We have enough checks for runnerForTask. So we do not need to
duplicate them when checking for StartTask.
While there, refactor how we start tasks to remove the need for
extra runner functions.
This is the objective I wanted to achieve for oonimkall:
1. less duplicate tests, and
2. more unit tests (which are less flaky)
At this point, we're basically done (pending forwardporting to
master) with https://github.com/ooni/probe/issues/1903.
* fix(oonimkall): TestStartTaskGood shouldn't cancel the test
This creates a race condition where the test may fail if we cannot
complete the whole "Example" test in less than one second.
This should explain the build failures I've seen so far and why
I didn't see those failures when running locally.
2021-12-02 12:47:07 +01:00
|
|
|
return dep.MockableInputPolicy()
|
|
|
|
}
|
|
|
|
|
|
|
|
func (dep *MockableTaskRunnerDependencies) NewExperimentInstance() taskExperiment {
|
|
|
|
if f := dep.MockableNewExperimentInstance; f != nil {
|
|
|
|
return f()
|
|
|
|
}
|
|
|
|
return dep
|
|
|
|
}
|
|
|
|
|
|
|
|
func (dep *MockableTaskRunnerDependencies) Interruptible() bool {
|
|
|
|
return dep.MockableInterruptible()
|
|
|
|
}
|
|
|
|
|
|
|
|
func (dep *MockableTaskRunnerDependencies) KibiBytesReceived() float64 {
|
|
|
|
return dep.MockableKibiBytesReceived()
|
|
|
|
}
|
|
|
|
|
|
|
|
func (dep *MockableTaskRunnerDependencies) KibiBytesSent() float64 {
|
|
|
|
return dep.MockableKibiBytesSent()
|
|
|
|
}
|
|
|
|
|
|
|
|
func (dep *MockableTaskRunnerDependencies) OpenReportContext(ctx context.Context) error {
|
|
|
|
return dep.MockableOpenReportContext(ctx)
|
|
|
|
}
|
|
|
|
|
|
|
|
func (dep *MockableTaskRunnerDependencies) ReportID() string {
|
|
|
|
return dep.MockableReportID()
|
|
|
|
}
|
|
|
|
|
|
|
|
func (dep *MockableTaskRunnerDependencies) MeasureWithContext(ctx context.Context, input string) (
|
|
|
|
measurement *model.Measurement, err error) {
|
|
|
|
return dep.MockableMeasureWithContext(ctx, input)
|
|
|
|
}
|
|
|
|
|
|
|
|
func (dep *MockableTaskRunnerDependencies) SubmitAndUpdateMeasurementContext(
|
|
|
|
ctx context.Context, measurement *model.Measurement) error {
|
|
|
|
return dep.MockableSubmitAndUpdateMeasurementContext(ctx, measurement)
|
|
|
|
}
|
|
|
|
|
|
|
|
// MockableKVStoreFSBuilder is a mockable taskKVStoreFSBuilder.
|
|
|
|
type MockableKVStoreFSBuilder struct {
|
|
|
|
MockNewFS func(path string) (model.KeyValueStore, error)
|
|
|
|
}
|
|
|
|
|
|
|
|
var _ taskKVStoreFSBuilder = &MockableKVStoreFSBuilder{}
|
|
|
|
|
|
|
|
func (m *MockableKVStoreFSBuilder) NewFS(path string) (model.KeyValueStore, error) {
|
|
|
|
return m.MockNewFS(path)
|
|
|
|
}
|