31cf7d2fdf
* doc: ensure all top dirs have an explanatory README This makes the repository a lil bit nicer to newcomers. Part of https://github.com/ooni/probe/issues/1335 * fix: re-run bindata to embed the README The readme is small, so we can pay the price of adding it. On a related note, I am very pleased the Go team implemented the `//go:embed` feature, so we can get rid of this bindata thing.
55 lines
1.9 KiB
Markdown
55 lines
1.9 KiB
Markdown
# Quality Assurance scripts
|
|
|
|
This directory contains quality assurance scripts that use Jafar to
|
|
ensure that OONI implementations behave. These scripts take on the
|
|
command line as argument the path to a binary with a OONI Probe v2.x
|
|
like command line interface. We do not care about full compatibility
|
|
but rather about having enough similar flags that running these tools
|
|
in parallel is not too much of a burden for us.
|
|
|
|
Tools with this shallow-compatible CLI are:
|
|
|
|
1. `github.com/ooni/probe-legacy`
|
|
2. `github.com/measurement-kit/measurement-kit/src/measurement_kit`
|
|
3. `github.com/ooni/probe-engine/internal/cmd/miniooni`
|
|
|
|
## Run QA on a Linux system
|
|
|
|
These scripts assume you're on a Linux system with `iptables`, `bash`,
|
|
`python3`, and possibly a bunch of other tools installed.
|
|
|
|
To start the QA script, run this command:
|
|
|
|
```bash
|
|
sudo ./QA/$nettest.py $ooni_exe
|
|
```
|
|
|
|
where `$nettest` is the nettest name (e.g. `telegram`) and `$ooni_exe`
|
|
is the OONI Probe v2.x compatible binary to test.
|
|
|
|
The Python script needs to run as root. Note however that sudo will also
|
|
be used to run `$ooni_exe` with the privileges of the `nobody` user.
|
|
|
|
## Run QA using a docker container
|
|
|
|
Run test in a suitable Docker container using:
|
|
|
|
```bash
|
|
./QA/rundocker.sh $nettest
|
|
```
|
|
|
|
Note that this will run a `--privileged` docker container. This will
|
|
eventually run the Python script you would run on Linux.
|
|
|
|
For now, the docker scripts only perform QA of `miniooni`.
|
|
|
|
## Diagnosing issues
|
|
|
|
The Python script that performs the QA runs a specific OONI test under
|
|
different failure conditions and stops at the first unexpected value found
|
|
in the resulting JSONL report. You can infer what went wrong by reading
|
|
the output of the `$ooni_exe` command itself, which should be above the point
|
|
where the Python script stopped, as well as by inspecting the JSONL file on
|
|
disk. By convention such file is named `$nettest.jsonl` and only contains
|
|
the result of the last run of `$nettest`.
|