ooni-probe-cli/internal/engine/experiment
Simone Basso b5da8be183
fix(netxlite): robust {ReadAll,Copy}Context with wrapped io.EOF (#661)
* chore(netxlite): add currently failing test case

This diff introduces a test cases that will fail because of the reason
explained in https://github.com/ooni/probe/issues/1965.

* chore(netxlite/iox_test.go): add failing unit tests

These tests directly show how the Go implementation of ReadAll
and Copy has the issue of checking for io.EOF equality.

* fix(netxlite): make {ReadAll,Copy}Context robust to wrapped io.EOF

The fix is simple: we just need to check for `errors.Is(err, io.EOF)`
after either io.ReadAll or io.Copy has returned. When this condition is
true, we need to convert the error back to `nil` as it ought to be.

While there, observe that the unit tests I committed in the previous
commit are wrongly asserting that the error must be wrapped. This
assertion is not correct, because in both cases we have just ensured
that the returned error is `nil` (i.e., success).

See https://github.com/ooni/probe/issues/1965.

* cleanup: remove previous workaround for wrapped io.EOF

These workarounds were partial, meaning that they would cover some
cases in which the issue occurred but not all of them.

Handling the problem in `netxlite.{ReadAll,Copy}Context` is the
right thing to do _as long as_ we always use these functions instead
of `io.{ReadAll,Copy}`.

This is why it's now important to ensure we clearly mention that
inside of the `CONTRIBUTING.md` guide and to also ensure that we're
not using these functions in the code base.

* fix(urlgetter): repair tests who assumed to see EOF error

Now that we have established that we should normalize EOF when
reading bodies like the stdlib does and now that it's clear why
our behavior diverged from the stdlib, we also need to repair
all the tests that assumed this incorrect behavior.

* fix(all): don't use io{,util}.{Copy,ReadAll}

* feat: add checks to ensure we don't use io.{Copy,ReadAll}

* doc(netxlite): document we know how to deal w/ wrapped io.EOF

* fix(nocopyreadall.bash): add exception for i/n/iox.go
2022-01-12 14:26:10 +01:00
..
dash fix: ensure experiments return nil when we want to submit (#654) 2022-01-07 13:17:20 +01:00
dnscheck cleanup: remove unnecessary legacy interfaces (#656) 2022-01-07 18:33:37 +01:00
example fix: ensure experiments return nil when we want to submit (#654) 2022-01-07 13:17:20 +01:00
fbmessenger refactor: interfaces and data types into the model package (#642) 2022-01-03 13:53:23 +01:00
hhfm cleanup: remove unnecessary legacy interfaces (#656) 2022-01-07 18:33:37 +01:00
hirl cleanup: remove unnecessary legacy interfaces (#656) 2022-01-07 18:33:37 +01:00
httphostheader refactor: interfaces and data types into the model package (#642) 2022-01-03 13:53:23 +01:00
ndt7 cleanup: remove unnecessary legacy interfaces (#656) 2022-01-07 18:33:37 +01:00
psiphon fix: ensure experiments return nil when we want to submit (#654) 2022-01-07 13:17:20 +01:00
riseupvpn refactor: interfaces and data types into the model package (#642) 2022-01-03 13:53:23 +01:00
run fix: ensure experiments return nil when we want to submit (#654) 2022-01-07 13:17:20 +01:00
signal refactor: interfaces and data types into the model package (#642) 2022-01-03 13:53:23 +01:00
sniblocking refactor: interfaces and data types into the model package (#642) 2022-01-03 13:53:23 +01:00
stunreachability cleanup: remove unnecessary legacy interfaces (#656) 2022-01-07 18:33:37 +01:00
telegram refactor: interfaces and data types into the model package (#642) 2022-01-03 13:53:23 +01:00
tlstool cleanup: remove unnecessary legacy interfaces (#656) 2022-01-07 18:33:37 +01:00
tor fix: ensure experiments return nil when we want to submit (#654) 2022-01-07 13:17:20 +01:00
torsf refactor: interfaces and data types into the model package (#642) 2022-01-03 13:53:23 +01:00
urlgetter fix(netxlite): robust {ReadAll,Copy}Context with wrapped io.EOF (#661) 2022-01-12 14:26:10 +01:00
webconnectivity cleanup: merge legacy errorsx in netxlite and hide classifiers (#655) 2022-01-07 17:31:21 +01:00
websteps fix(netxlite): robust {ReadAll,Copy}Context with wrapped io.EOF (#661) 2022-01-12 14:26:10 +01:00
webstepsx cleanup: remove redundant HTTPClient definition (#643) 2022-01-03 16:47:54 +01:00
whatsapp refactor: interfaces and data types into the model package (#642) 2022-01-03 13:53:23 +01:00
README.md chore: merge probe-engine into probe-cli (#201) 2021-02-02 12:05:47 +01:00

Directory github.com/ooni/probe-engine/experiment

This directory contains the implementation of all the supported experiments, one for each directory. The OONI spec repository contains a description of all the specified experiments.

Note that in the OONI spec repository experiments are called nettests. Originally, they were also called nettests here but that created confusion with nettests in ooni/probe-cli. Therefore, we now use the term experiment to indicate the implementation and the term nettest to indicate the user facing view of such implementation.

Note that some experiments implemented here are not part of the OONI specification. For example, the urlgetter experiment is not in the OONI spec repository. The reason why this happens is that urlgetter is an experiment "library" that other experiments use to implement their functionality.

Likewise, the example experiment is a minimal experiment that does nothing and you could use to bootstrap the implementation of a new experiment. Of course, this experiment is not part of the OONI specification.