* go.mod go.sum: pin to ooni/probe-engine@v0.13.0, update all other deps
Make sure dependencies of probe-engine are at the same exact version to
which is probe-engine, to avoid any possible issue.
* Readme.md: clarify the procedure for updating deps
* nettests/groups.go: remove redundant struct names
* go.mod go.sum: update deps except probe-engine
* Update to ooni/probe-engine@e768161f91
The API has changed. Methods that used to change bits of the session have
been removed. Now the session is more immutable than before.
As such, we need to completely fill the config before using it.
* Set IncludeCountry to always true
Co-authored-by: Arturo Filastò <arturo@filasto.net>
Closes https://github.com/ooni/probe-engine/issues/316
Rationale: a measurement may fail for a bunch of reasons including
buts in the code. The original determination was that we won't
return an error from the measurement in case of anomaly. But doing
that is error prone, and indeed the Psiphon experiment was not
following this pattern. I claim that this pattern was wrong, and
it's much more in our interest to submit and store on disk anything
that we get from a measurement. This data can be useful to look
into bugs, as well as to discover novel anomalies we didnt' anticipate.
* go.mod go.sum: update all non-probe-engine deps
For each line in the go.mod, run `go get -u -v $package` if the
line is not an indirect dependency and is not probe-engine.
Upgrading probe-engine is going to require the same spell that
is used in probe-engine to update psiphon.
* go get -v github.com/ooni/probe-engine@v0.5.0
This just pins to the latest probe-engine but we've not manually
pinned all the other dependencieds yet.
Take care of the trivial API changes in probe-engine as well, such
that we can have a working build after this commit.
* go.mod go.sum: pin to probe-engine dependencies
Basically: remove all indirect dependencies. Merge this go.mod with
the one of probe-engine, to pin dependencies. Run `go mod tidy`.
* circumvention: add basic implementation of tor
This needs to be polished further, of course. But at least we have
now added support for running tor in the circumvention group.
* Readme.md: document how to update dependencies
* go get -v github.com/ooni/probe-engine@fcc9ee0a7afb
* go get -v github.com/ooni/probe-engine@4d254f5b2
* nettests/tor.go: implement summary test keys
1. only print time left if ETA is positive
2. skip ETA calculation with a single input
3. don't compute ETA for first entry[*]
[*] this is actually what avoids emitting infinite but the other
parts of this diff felt useful yak shaving as well.
Closes#91
* Remove all settings which are not implemented
* Add support for migrating config files from version 0 -> 1
Add basic unittests for the config file migration
* Update the config file used by travis
* Fix typos in console log messages
* Improve the comment
* Fix type of limit
* Set informed consent to true in the travis config
* utils/geoip.go: use github.com/ooni/probe-engine
Let's start using the engine by rewriting utils/geoip.go to
be just a thin wrapper around the engine functionality.
* Ready for review
* Checkpoint: the im tests are converted
Still have some doubts with respect to the variables that
are passed to MK via probe-engine. Will double check.
* fix(i/c/r/run.go): write the correct logic
* nettests: one more comment and also fix a format string
* Tweak previous
* progress
* Fix doofus
* better comment
* XXX => actionable comment
* Add glue to simplify test keys management
Making the concept of measurement more abstract in the engine is
not feasible because, when submitting a measurement, we need to
modify it to update the report ID and the measurement ID. Therefore,
returning a serialized measurement is not a good idea. We will
keep using a model.Measurement in the engine.
Changing model.Measurement.TestKeys's type from a `interface{}`
pointing to a well defined data structure to `map[string]interface{}`
is a regression because means that we are moving from code that
has a clear and defined structure to code that is more complicated
to parse and validate. Since we're already suffering havily from
the lack of a good schema, I'm not going to make the situation
worst by worsening the engine. At least for ndt7 and psiphon, we
now have a good schema and I don't want to lose that.
However, the current code in this repository is expecting the
test keys to be a `map[string]interface{}`. This choice was
dictated by the fact that we receive a JSON from Measurement Kit
and by the fact that there's not a clear schema.
To solve this tension, in this commit I am going to write glue
adapter code that makes sure that the TestKeys of a Measurement
are converted to `map[string]interface{}`. This will be done
using a type cast where possible and JSON serialization and parsing
otherwise. In a perfect world, glue is not a good idea, but in a
real world it may actually be useful.
When all tests in the engine will have a clear Go data structure,
we'll then remove the glue and just cast to the proper data
structure from `interface{}` where required.
* nettests/performance: use probe-engine
* go.{mod,sum}: upgrade to latest probe-engine
* nettests/middlebox: use ooni/probe-engine
* Update to the latest probe-engine
* web_connectivity: rewrite to use probe-engine
* Cosmetic change suggested by @hellais
* nettests/nettests.go: remove unused code
* nettests/nettests.go: fix progress
* nettests/nettests.go: remove go-measurement-kit code
* We don't depend on go-measurement-kit anymore
* Improve non-verbose output where possible
See also: https://github.com/measurement-kit/measurement-kit/issues/1856
* Make web_connectivity output pleasant
* Update to the latest probe-engine
* nettests/nettests.go: honour sharing settings
* Update to the latest probe-engine
* Use log.WithFields for probe-engine
* Update go.mod go.sum
* Revert "Update go.mod go.sum"
This reverts commit 5ecd38d8236f4a4e9b77ddb8e8a0d1e3cdd4b818.
* Revert "Revert "Update go.mod go.sum""
This reverts commit 6114b31eca98826112032776bd0feff02d763ecd.
* Upgrade ooni/probe-engine
* Unset GOPATH before running go build commands
* Dockefile: fix linux build by using latest
* Update to the latest ooni/probe-engine
```
go get -u github.com/ooni/probe-engine
go mod tidy
```
* Repair build
* Gopkg.lock: use MK v0.10.3
* ooni: stop using legacy GeoIP database files
* Some yak shaving of Makefile
1. remove now broken commands to download deps
2. also define the CXX cross compiler
* chore(dep): migrate from dep to go 1.11+ modules
See https://blog.callr.tech/migrating-from-dep-to-go-1.11-modules/
I need this to simplify my life in building for Travis.
* Introduce build.sh and repair build
In going forward, I believe we don't actually need a Makefile but I
didn't want to make such a radical change now.
* Another strategy wrt gopath
* travis: run regress tests on macOS
Closes#30
