e72263dacb
This diff forward ports 36ba3630c9002db0bd79e3a7e49641ce6b665471, whose original commit message follows: - - - This diff contains minimal changes to make webconnectivity QA WAI with the new Web Connectivity test helper. It seems we're currently doing round robin between the old and the new implementation, so I needed to locally pin my probes to use the new implementation by changing the code. But, obviously, I don't want to commit this code. Likewise, in my working environment, I need to build the docker container using `docker buildx build --platform linux/amd64`, but I am not sure whether to commit this code. While there, I noticed there was a missing QA test for the case in which we're passing through a transparent HTTP proxy. I noticed as well that the test that said it was passing through such a proxy was actually using a transparent TLS proxy. I remediated this by ensuring we have a test for both cases. The other major change in the suite is that, when using the new TH, there's uncommon headers intersection in some tests, so we have had a flip from headers not matching to headers matching. Finally, some formatting changes because I did re-run black. These changes should be enough to call it a day with respect to QA (see https://github.com/ooni/probe/issues/2016#issuecomment-1033813344). This diff WILL need to be forward ported to master. (I don't know whether the GitHub QA will converge after these changes and I suspect it won't because of the test helper round robin.) |
||
|---|---|---|
| .. | ||
| lib | ||
| .dockerignore | ||
| .gitignore | ||
| common.py | ||
| Dockerfile | ||
| fbmessenger.py | ||
| hhfm.py | ||
| hirl.py | ||
| index.mjs | ||
| minioonilike.py | ||
| probeasn.py | ||
| pyrun.sh | ||
| README.md | ||
| rundocker.bash | ||
| telegram.py | ||
| webconnectivity.py | ||
| whatsapp.py | ||
Quality Assurance scripts
This directory contains quality assurance scripts that use Jafar to ensure that OONI implementations behave. These scripts take on the command line as argument the path to a binary with a OONI Probe v2.x like command line interface. We do not care about full compatibility but rather about having enough similar flags that running these tools in parallel is not too much of a burden for us.
Tools with this shallow-compatible CLI are:
github.com/ooni/probe-legacygithub.com/measurement-kit/measurement-kit/src/measurement_kitgithub.com/ooni/probe-engine/internal/cmd/miniooni
Run QA on a Linux system
These scripts assume you're on a Linux system with iptables, bash,
python3, and possibly a bunch of other tools installed.
To start the QA script, run this command:
sudo ./QA/$nettest.py $ooni_exe
where $nettest is the nettest name (e.g. telegram) and $ooni_exe
is the OONI Probe v2.x compatible binary to test.
The Python script needs to run as root. Note however that sudo will also
be used to run $ooni_exe with the privileges of the nobody user.
Run QA using a docker container
Run test in a suitable Docker container using:
./QA/rundocker.sh $nettest
Note that this will run a --privileged docker container. This will
eventually run the Python script you would run on Linux.
For now, the docker scripts only perform QA of miniooni.
Diagnosing issues
The Python script that performs the QA runs a specific OONI test under
different failure conditions and stops at the first unexpected value found
in the resulting JSONL report. You can infer what went wrong by reading
the output of the $ooni_exe command itself, which should be above the point
where the Python script stopped, as well as by inspecting the JSONL file on
disk. By convention such file is named $nettest.jsonl and only contains
the result of the last run of $nettest.