b7cc309901
This diff re-implements the vanilla_tor experiment. This experiment was part of the ooni/probe-legacy implementation. The reference issue is https://github.com/ooni/probe/issues/803. We didn't consider the possible improvements mentioned by the https://github.com/ooni/probe/issues/803#issuecomment-598715694 comment, which means we'll need to create a follow-up issue for them. We will then decide whether, when, and how to implement those follow-up measurements either into `vanilla_tor` or into the existing `tor` experiment. This novel `vanilla_tor` implementation emits test_keys that are mostly compatible with the original implementation, however: 1. the `timeout` is a `float64` rather than integer (but the default timeout is an integer, so there are no JSON-visible changes); 2. the `tor_log` string is gone and replaced by the `tor_logs` list of strings, which contains the same information; 3. the definition of `error` has been augmented to include the case in which there is an unknown error; 4. the implementation of vanilla_tor mirrors closely the one of torsf and we have taken steps to make the two implementations as comparable as possible in terms of the generated JSON measurement. The main reason why we replaced `tor_log` with `tor_logs` are: 1. that `torsf` already used that; 2. that reading the JSON is easier with this implementation compared to an implementation where all logs are into the same string. If one is processing the new data format using Python, then it will not be difficult convert `tor_log` to `tor_logs`. In any case, because we extract the most interesting fields (e.g., the percentage of the bootstrap where tor fails), it seems that logs are probably more useful as something you want to read in edge cases (I guess). Also, because we want `torsf` and `vanilla_tor` to have similar JSONs, we renamed `torsf`'s `default_timeout` to `timeout`. This change has little to none real-world impact, because no stable version of OONI Probe has ever shipped a `torsf` producing the `default_timeout` field. Regarding the structure of this diff, we have: 1. factored code to parse tor logs into a separate package; 2. implemented `vanilla_tor` as a stripped down `torsf` and added further changes to ensure compatibility with the previous `vanilla_tor`'s data format; 3. improved `torsf` to merge back the changes in `vanilla_tor`, so the two data formats of the two experiments are as similar as possible. We believe producing as similar as possible data formats helps anyone who's reading measurements generated by both experiments. We have retained/introduced `vanilla_tor`'s `error` field, which is not very useful when one has a more precise failure but is still what `vanilla_tor` used to emit, so it makes sense to also have this field. In addition to changing the implementation, we also updated the specs. As part of our future work, we may want to consider factoring the common code of these two experiments into the same underlying support library. |
||
|---|---|---|
| .. | ||
| dash | ||
| dnscheck | ||
| dnsping | ||
| example | ||
| fbmessenger | ||
| hhfm | ||
| hirl | ||
| httphostheader | ||
| ndt7 | ||
| psiphon | ||
| quicping | ||
| riseupvpn | ||
| run | ||
| signal | ||
| simplequicping | ||
| sniblocking | ||
| stunreachability | ||
| tcpping | ||
| telegram | ||
| tlsping | ||
| tlstool | ||
| tor | ||
| torsf | ||
| urlgetter | ||
| vanillator | ||
| webconnectivity | ||
| websteps | ||
| webstepsx | ||
| README.md | ||
Directory github.com/ooni/probe-engine/experiment
This directory contains the implementation of all the supported experiments, one for each directory. The OONI spec repository contains a description of all the specified experiments.
Note that in the OONI spec repository experiments are called nettests. Originally, they were also called nettests here but that created confusion with nettests in ooni/probe-cli. Therefore, we now use the term experiment to indicate the implementation and the term nettest to indicate the user facing view of such implementation.
Note that some experiments implemented here are not part of
the OONI specification. For example, the urlgetter
experiment is not in the OONI spec repository. The reason why
this happens is that urlgetter is an experiment "library" that
other experiments use to implement their functionality.
Likewise, the example experiment is a minimal experiment that does nothing and you could use to bootstrap the implementation of a new experiment. Of course, this experiment is not part of the OONI specification.