While there, include integration testing to make sure the script
is working as intended before using it.
While there, edit maketarball.bash's comments.
This diff forward ports 36ba3630c9002db0bd79e3a7e49641ce6b665471,
whose original commit message follows:
- - -
This diff contains minimal changes to make webconnectivity QA
WAI with the new Web Connectivity test helper.
It seems we're currently doing round robin between the old and
the new implementation, so I needed to locally pin my probes
to use the new implementation by changing the code. But, obviously,
I don't want to commit this code.
Likewise, in my working environment, I need to build the docker
container using `docker buildx build --platform linux/amd64`, but
I am not sure whether to commit this code.
While there, I noticed there was a missing QA test for the case
in which we're passing through a transparent HTTP proxy. I noticed
as well that the test that said it was passing through such a
proxy was actually using a transparent TLS proxy. I remediated
this by ensuring we have a test for both cases.
The other major change in the suite is that, when using the new TH,
there's uncommon headers intersection in some tests, so we have
had a flip from headers not matching to headers matching.
Finally, some formatting changes because I did re-run black.
These changes should be enough to call it a day with respect to
QA (see https://github.com/ooni/probe/issues/2016#issuecomment-1033813344).
This diff WILL need to be forward ported to master.
(I don't know whether the GitHub QA will converge after these changes
and I suspect it won't because of the test helper round robin.)
This pull request introduces a set of Node.js scripts for performing A/B comparison of websteps and webconnectivity as described in https://github.com/ooni/probe/issues/1805. Rather than using Jafar, I ended up using `miniooni`'s `--censor` command line flag introduced in [v3.12.0-alpha.1](https://github.com/ooni/probe-cli/releases/tag/v3.12.0-alpha.1). The main reason for doing so is that it's simpler to run tests without requiring root access and Linux _and_ Docker (e.g., I did not develop part of this diff using Linux). Additionally, I choose to use Node.js rather than extending the existing Python framework for QA, because I found Node.js easier when working with JSON data.
The issue at https://github.com/ooni/probe/issues/1741 is that running `docker --platform linux/arm64` segfaults when running `sudo apt-get update -q` inside the `arm64` docker environment.
As far as the `debianrepo` rule is concerned, we can fix the issue by taking advantage of Debian multi-arch. We now configure Debian multi-arch and install the package inside a `debian:stable` environment.
We keep using docker. In principle we could not. But the Ubuntu environment provided by GitHub actions does not support multi-arch for arm. Also, I'd like testing this rule to be possible also locally (where I don't have Debian).
The current implementation assumes the user has already installed tor
on the current system. If tor is not present, the experiment fails.
This is meant to be the first version of this experiment.
We are going to add more functionality in subsequent revisions of
this experiment, once we've collected more feedback.
Reference issue: https://github.com/ooni/probe/issues/1565.
Here's the spec PR: https://github.com/ooni/spec/pull/218.
Here's the issue tracking future work: https://github.com/ooni/probe/issues/1686
We are mostly good to declare a stable release. We still need to deal with https://github.com/ooni/probe/issues/1484.
In this PR, we fix the aforementioned issue. These are the changes:
1. we remove the vendored `debops-ci`, and we pull it directly from `ooni/sysadmin`
2. we introduce a new script, `./CLI/linux/pubdebian`, to publish packages
3. we modify `./mk` to allow for publishing debian packages built outside of CI
The latter point has been quite useful in debugging what was wrong.
1. we can merge the e2eminiooni.yml test into the miniooni.yml test
so to reduce the number of tests we run;
2. ideally we would like the smoketest.sh test to evolve and also
check whether we can fetch the measurements we submitted, so start
moving this script into the `./E2E` folder, add a note saying we
would like to do that, and direct all the tests to run this script
at its new location and with its new name (`ooniprobe.sh`).
With these two changes, it's fine to remove the ooniprobe2debian.yml
test in ooni/e2etesting because we're moving its functionality to this
repository. (We mentioned the need to do this move in a previous TODO
comment at the top of such a script.)
Work part of https://github.com/ooni/probe/issues/1468
Part of https://github.com/ooni/probe/issues/1466. We're building both `arm64` and `amd64`. We are still not publishing `arm64` packages, which is what is asked in the original issue, but we're really close to doing that.
Part of https://github.com/ooni/probe/issues/1440.
Basically, let us continue to update our build infrastructure so that we can release v3.10.0-beta.
Now, it's the turn of iOS.
* refactor: move more commands to internal/cmd
Part of https://github.com/ooni/probe/issues/1335.
We would like all commands to be at the same level of engine
rather than inside engine (now that we can do it).
* fix: update .gitignore
* refactor: also move jafar outside engine
* We should be good now?
* refactor: miniooni should be outside of the engine
This is part of https://github.com/ooni/probe/issues/1335. We also need
to think whether we wanna keep libminiooni and miniooni separated.
The previous use case for having a top-level libminiooni was that of
enabling others to integrate miniooni into other binaries.
This was usegul when studying internet censorship in Spain in May 2020.
I am wondering whether we should be keeping this complexity. I am not
sure about this and probably we should be killing it.
(In any case, reducing complexity is not the objective of this diff,
since I would like instead to move things around with minimal changes
and make sure we have a ~good repository organization here.)
* fix: import in libminiooni
* Use ooni/probe-engine 0.16.0
* Update all the other dependencies
* Use GitHub Actions rather than Travis CI
* Automatically build and test binaries on the target OS (for Windows, macOS, Linux on amd64)
* Make sure we correctly measure coverage
* Make sure we use `-race` when running tests
* Remove unnecessary scripts
* Make sure the README is up-to-date
* Write small script to update binary data and add GitHub Actions checks for it
* Notice that we needed to run ./updatebindata.sh and run it
* Self documenting instructions regarding cross compiling
* Set version number to v3.0.7-beta
Part of https://github.com/ooni/probe-engine/issues/748
* Gopkg.lock: use MK v0.10.3
* ooni: stop using legacy GeoIP database files
* Some yak shaving of Makefile
1. remove now broken commands to download deps
2. also define the CXX cross compiler
* chore(dep): migrate from dep to go 1.11+ modules
See https://blog.callr.tech/migrating-from-dep-to-go-1.11-modules/
I need this to simplify my life in building for Travis.
* Introduce build.sh and repair build
In going forward, I believe we don't actually need a Makefile but I
didn't want to make such a radical change now.
* Another strategy wrt gopath
* travis: run regress tests on macOS
Closes#30
