ooni-probe-cli/internal/engine/experiment
Simone Basso 85664f1e31
feat(torsf): collect tor logs, select rendezvous method, count bytes (#683)
This diff contains significant improvements over the previous
implementation of the torsf experiment.

We add support for configuring different rendezvous methods after
the convo at https://github.com/ooni/probe/issues/2004. In doing
that, I've tried to use a terminology that is consistent with the
names being actually used by tor developers.

In terms of what to do next, this diff basically instruments
torsf to always rendezvous using domain fronting. Yet, it's also
possible to change the rendezvous method from the command line,
when using miniooni, which allows to experiment a bit more. In the
same vein, by default we use a persistent tor datadir, but it's
also possible to use a temporary datadir using the cmdline.

Here's how a generic invocation of `torsf` looks like:

```bash
./miniooni -O DisablePersistentDatadir=true \
           -O RendezvousMethod=amp \
           -O DisableProgress=true \
           torsf
```

(The default is `DisablePersistentDatadir=false` and
`RendezvousMethod=domain_fronting`.)

With this implementation, we can start measuring whether snowflake
and tor together can boostrap, which seems the most important thing
to focus on at the beginning. Understanding why the bootstrap most
often does not converge with a temporary datadir on Android devices
remains instead an open problem for now. (I'll also update the
relevant issues or create new issues after commit this.)

We also address some methodology improvements that were proposed
in https://github.com/ooni/probe/issues/1686. Namely:

1. we record the tor version;

2. we include the bootstrap percentage by reading the logs;

3. we set the anomaly key correctly;

4. we measure the bytes send and received (by `tor` not by `snowflake`, since
doing it for snowflake seems more complex at this stage).

What remains to be done is the possibility of including Snowflake
events into the measurement, which is not possible until the new
improvements at common/event in snowflake.git are included into a
tagged version of snowflake itself. (I'll make sure to mention
this aspect to @cohosh in https://github.com/ooni/probe/issues/2004.)
2022-02-07 17:05:36 +01:00
..
dash feature: merge measurex and netx archival layer (1/N) (#663) 2022-01-14 12:13:10 +01:00
dnscheck cleanup: remove unnecessary legacy interfaces (#656) 2022-01-07 18:33:37 +01:00
example fix: ensure experiments return nil when we want to submit (#654) 2022-01-07 13:17:20 +01:00
fbmessenger refactor: interfaces and data types into the model package (#642) 2022-01-03 13:53:23 +01:00
hhfm cleanup: remove unnecessary legacy interfaces (#656) 2022-01-07 18:33:37 +01:00
hirl cleanup: remove unnecessary legacy interfaces (#656) 2022-01-07 18:33:37 +01:00
httphostheader refactor: interfaces and data types into the model package (#642) 2022-01-03 13:53:23 +01:00
ndt7 cleanup: remove unnecessary legacy interfaces (#656) 2022-01-07 18:33:37 +01:00
psiphon fix: ensure experiments return nil when we want to submit (#654) 2022-01-07 13:17:20 +01:00
riseupvpn refactor: interfaces and data types into the model package (#642) 2022-01-03 13:53:23 +01:00
run fix: ensure experiments return nil when we want to submit (#654) 2022-01-07 13:17:20 +01:00
signal refactor: interfaces and data types into the model package (#642) 2022-01-03 13:53:23 +01:00
sniblocking refactor: interfaces and data types into the model package (#642) 2022-01-03 13:53:23 +01:00
stunreachability cleanup: remove unnecessary legacy interfaces (#656) 2022-01-07 18:33:37 +01:00
telegram refactor: interfaces and data types into the model package (#642) 2022-01-03 13:53:23 +01:00
tlstool cleanup: remove unnecessary legacy interfaces (#656) 2022-01-07 18:33:37 +01:00
tor fix: ensure experiments return nil when we want to submit (#654) 2022-01-07 13:17:20 +01:00
torsf feat(torsf): collect tor logs, select rendezvous method, count bytes (#683) 2022-02-07 17:05:36 +01:00
urlgetter feat(torsf): collect tor logs, select rendezvous method, count bytes (#683) 2022-02-07 17:05:36 +01:00
webconnectivity cleanup: merge legacy errorsx in netxlite and hide classifiers (#655) 2022-01-07 17:31:21 +01:00
websteps fix(netxlite): robust {ReadAll,Copy}Context with wrapped io.EOF (#661) 2022-01-12 14:26:10 +01:00
webstepsx cleanup: remove redundant HTTPClient definition (#643) 2022-01-03 16:47:54 +01:00
whatsapp refactor: interfaces and data types into the model package (#642) 2022-01-03 13:53:23 +01:00
README.md chore: merge probe-engine into probe-cli (#201) 2021-02-02 12:05:47 +01:00

Directory github.com/ooni/probe-engine/experiment

This directory contains the implementation of all the supported experiments, one for each directory. The OONI spec repository contains a description of all the specified experiments.

Note that in the OONI spec repository experiments are called nettests. Originally, they were also called nettests here but that created confusion with nettests in ooni/probe-cli. Therefore, we now use the term experiment to indicate the implementation and the term nettest to indicate the user facing view of such implementation.

Note that some experiments implemented here are not part of the OONI specification. For example, the urlgetter experiment is not in the OONI spec repository. The reason why this happens is that urlgetter is an experiment "library" that other experiments use to implement their functionality.

Likewise, the example experiment is a minimal experiment that does nothing and you could use to bootstrap the implementation of a new experiment. Of course, this experiment is not part of the OONI specification.