* feat: use go1.16 embedding for resources
We want to embed everything that can be easily embedded. We should, at a
minimum, replace the downloading of resources and bindata.
Ref: https://github.com/ooni/probe/issues/1367.
* fix: get rid of bindata and use go embed instead
* fix: start unbreaking some automatic tests
* fix: fetch resources as part of the mobile build
* fix: convert more stuff to go1.16
I still expect many breakages, but we'll fix them.
* fix: make the windows CI green
* fix: get resources before running QA
* fix: go1.16 uses modules by default
* hopefully fix all other outstanding issues
* fix(QA/telegram.py): add another DC IP address
* Apply suggestions from code review
This is how I did it:
1. `git clone https://github.com/ooni/probe-engine internal/engine`
2. ```
(cd internal/engine && git describe --tags)
v0.23.0
```
3. `nvim go.mod` (merging `go.mod` with `internal/engine/go.mod`
4. `rm -rf internal/.git internal/engine/go.{mod,sum}`
5. `git add internal/engine`
6. `find . -type f -name \*.go -exec sed -i 's@/ooni/probe-engine@/ooni/probe-cli/v3/internal/engine@g' {} \;`
7. `go build ./...` (passes)
8. `go test -race ./...` (temporary failure on RiseupVPN)
9. `go mod tidy`
10. this commit message
Once this piece of work is done, we can build a new version of `ooniprobe` that
is using `internal/engine` directly. We need to do more work to ensure all the
other functionality in `probe-engine` (e.g. making mobile packages) are still WAI.
Part of https://github.com/ooni/probe/issues/1335
* feat: sketch out periodic command
* feat: sketch out periodic command for macOS
* feat: implement darwin's launch agent
* refactor: better way to run on darwin
Make sure we have code that builds on all platforms.
* fix(run): max 10 URLs with darwin in unattended mode
* feat: add support for seeing/streaming logs
* feat: implement the status command and add usage hints
* feat(periodic): run onboarding if needed
* fix: no too confusing function names
* fix: s/periodic/autorun/
Discussed earlier this morning with @hellais.
* fix: we cannot show logs before Big Sur
Bug reported by @hellais.
The CloseReport method is gone. We don't need to close reports
anymore with the new OONI backend.
The InputsRequired flags now is InputsOrQueryTestLists.
This diff pins to ooni/probe-engine@3049779878
and starts using the recently introduced probe-engine APIs.
Namely, here, we use the InputLoader for loading URLs.
I've confirmed manually everything is still working as intended.
Part of https://github.com/ooni/probe/issues/1283.
(In particular, the InputLoader is the abstraction allowing us to load
input from several sources, including command line flags and external
files.)
* feat: use ooni/probe-engine@286613b74e and cleanup
1. zap unused configuration settings from the config file but do not
bump the version number because doing that _may_ interact in unexpected
ways with probe-desktop (hence https://github.com/ooni/probe/issues/1297)
and also because we've just _removed_ stuff for now, therefore any
previous configuration file will continue to work, except that we'll
be ignoring a bunch of options. In a future version of probe-cli I'll
spend some time to further improve config file management.
2. accordingly, make sure all current configuration files that are around
in the tree are current and only feature supported options.
3. update to ooni/probe-engine@286613b74e, which contains a bunch of
APIs that should allow us to simplify the interaction between the cli and
the engine, by sharing code more cleverly.
4. zap GetTestKeys because now we use code in probe-engine instead.
5. zap LogSummary because it was not being used.
6. the main change related to cleaning up the config and to the update
to the latest probe-engine is that include_{cc,asn,ip} settings are
gone and we now share the CC and the ASN and we never share the IP addr.
Reference issue: https://github.com/ooni/probe/issues/1283.
After this change is landed, there's a bunch more work to do to further
unify cli and engine. The final state will be that the cli uses ~the code
used by miniooni, so it will have a bunch of desirable options.
* fix: bindata after recent changes
* chore: set version to 3.0.12-alpha
I need to bless 3.0.11 now to pin to ooni/probe-engine v0.20.2.
* chore: update all dependencies
Most notably, pin to ooni/probe-engine 0.20.2.
This beta release incorporates recent fixes in ooni/probe-engine that we
would like to test and ship relatively soon.
I'm doing this as part of https://github.com/ooni/explorer/issues/495
* Update go-bindata and regenerate binary data
* Pin to ooni/probe-engine 0.17.0 and update dependencies
* Set version to 3.0.7
* Readme.md: better release instructions
* Use ooni/probe-engine 0.16.0
* Update all the other dependencies
* Use GitHub Actions rather than Travis CI
* Automatically build and test binaries on the target OS (for Windows, macOS, Linux on amd64)
* Make sure we correctly measure coverage
* Make sure we use `-race` when running tests
* Remove unnecessary scripts
* Make sure the README is up-to-date
* Write small script to update binary data and add GitHub Actions checks for it
* Notice that we needed to run ./updatebindata.sh and run it
* Self documenting instructions regarding cross compiling
* Set version number to v3.0.7-beta
Part of https://github.com/ooni/probe-engine/issues/748
* go.mod go.sum: pin to ooni/probe-engine@v0.13.0, update all other deps
Make sure dependencies of probe-engine are at the same exact version to
which is probe-engine, to avoid any possible issue.
* Readme.md: clarify the procedure for updating deps
* nettests/groups.go: remove redundant struct names
* go.mod go.sum: update deps except probe-engine
* Update to ooni/probe-engine@e768161f91
The API has changed. Methods that used to change bits of the session have
been removed. Now the session is more immutable than before.
As such, we need to completely fill the config before using it.
* Set IncludeCountry to always true
Co-authored-by: Arturo Filastò <arturo@filasto.net>
I'm pinning to ooni/probe-engine@72c12b0b81
because it generates more correct data and I think we're going to have
most likely this probe-cli version in the stable desktop release.
We're specifically pinning a commit of probe-engine that is using
MK v0.10.11, so that we address the following issues:
1. we use db-ip.com for the country database as opposed to using
the increasingly stale MaxMind database (see https://github.com/ooni/probe-engine/issues/334)
2. we're using an implementation of WhatsApp that does not suffer
from the super-old CIDR bug (see https://github.com/ooni/probe-engine/issues/341)
3. we're not linking to libcurl anymore on Windows and Linux, thanks to
this new version of MK where we can optionally disable libcurl; we are
still linking to libcurl on macOS, but that has no impact on the binary
size since on macOS libcurl is part of the system
This should be enough, from my side to bless a new release of the
probe-cli (see https://github.com/ooni/probe/issues/1028).
At this stage is fine to just pin to the latest master. I am planning on
blessing periodic releases and it's fine to pin sometimes.
I'm pinning this specific version because it currently supports the
summary for experiment/tor, which @sarathms would benefit from.
* go.mod go.sum: update all non-probe-engine deps
For each line in the go.mod, run `go get -u -v $package` if the
line is not an indirect dependency and is not probe-engine.
Upgrading probe-engine is going to require the same spell that
is used in probe-engine to update psiphon.
* go get -v github.com/ooni/probe-engine@v0.5.0
This just pins to the latest probe-engine but we've not manually
pinned all the other dependencieds yet.
Take care of the trivial API changes in probe-engine as well, such
that we can have a working build after this commit.
* go.mod go.sum: pin to probe-engine dependencies
Basically: remove all indirect dependencies. Merge this go.mod with
the one of probe-engine, to pin dependencies. Run `go mod tidy`.
* circumvention: add basic implementation of tor
This needs to be polished further, of course. But at least we have
now added support for running tor in the circumvention group.
* Readme.md: document how to update dependencies
* go get -v github.com/ooni/probe-engine@fcc9ee0a7afb
* go get -v github.com/ooni/probe-engine@4d254f5b2
* nettests/tor.go: implement summary test keys
