ooni-probe-cli/internal
Simone Basso b5da8be183
fix(netxlite): robust {ReadAll,Copy}Context with wrapped io.EOF (#661)
* chore(netxlite): add currently failing test case

This diff introduces a test cases that will fail because of the reason
explained in https://github.com/ooni/probe/issues/1965.

* chore(netxlite/iox_test.go): add failing unit tests

These tests directly show how the Go implementation of ReadAll
and Copy has the issue of checking for io.EOF equality.

* fix(netxlite): make {ReadAll,Copy}Context robust to wrapped io.EOF

The fix is simple: we just need to check for `errors.Is(err, io.EOF)`
after either io.ReadAll or io.Copy has returned. When this condition is
true, we need to convert the error back to `nil` as it ought to be.

While there, observe that the unit tests I committed in the previous
commit are wrongly asserting that the error must be wrapped. This
assertion is not correct, because in both cases we have just ensured
that the returned error is `nil` (i.e., success).

See https://github.com/ooni/probe/issues/1965.

* cleanup: remove previous workaround for wrapped io.EOF

These workarounds were partial, meaning that they would cover some
cases in which the issue occurred but not all of them.

Handling the problem in `netxlite.{ReadAll,Copy}Context` is the
right thing to do _as long as_ we always use these functions instead
of `io.{ReadAll,Copy}`.

This is why it's now important to ensure we clearly mention that
inside of the `CONTRIBUTING.md` guide and to also ensure that we're
not using these functions in the code base.

* fix(urlgetter): repair tests who assumed to see EOF error

Now that we have established that we should normalize EOF when
reading bodies like the stdlib does and now that it's clear why
our behavior diverged from the stdlib, we also need to repair
all the tests that assumed this incorrect behavior.

* fix(all): don't use io{,util}.{Copy,ReadAll}

* feat: add checks to ensure we don't use io.{Copy,ReadAll}

* doc(netxlite): document we know how to deal w/ wrapped io.EOF

* fix(nocopyreadall.bash): add exception for i/n/iox.go
2022-01-12 14:26:10 +01:00
..
atomicx doc: cleanup and improve for recently moved pkgs (#354) 2021-06-04 11:39:00 +02:00
bytecounter refactor: interfaces and data types into the model package (#642) 2022-01-03 13:53:23 +01:00
cmd cleanup(netx): remove the DNSClient type (#660) 2022-01-10 11:53:06 +01:00
engine fix(netxlite): robust {ReadAll,Copy}Context with wrapped io.EOF (#661) 2022-01-12 14:26:10 +01:00
fakefill feat(httpx): improve testing using the fakefiller (#649) 2022-01-05 14:49:31 +01:00
fsx refactor: merge dnsx and errorsx into netxlite (#517) 2021-09-28 12:42:01 +02:00
httpx refactor: move httpx into the internal package (#646) 2022-01-05 17:17:20 +01:00
humanize fix(all): introduce and use iox.CopyContext (#380) 2021-06-15 13:44:28 +02:00
kvstore refactor: interfaces and data types into the model package (#642) 2022-01-03 13:53:23 +01:00
measurex fix(netxlite): robust {ReadAll,Copy}Context with wrapped io.EOF (#661) 2022-01-12 14:26:10 +01:00
mlablocate cleanup: remove redundant HTTPClient definition (#643) 2022-01-03 16:47:54 +01:00
mlablocatev2 cleanup: remove redundant HTTPClient definition (#643) 2022-01-03 16:47:54 +01:00
model refactor: move i/netx/archival structs to i/model (#659) 2022-01-10 11:25:52 +01:00
multierror doc: cleanup and improve for recently moved pkgs (#354) 2021-06-04 11:39:00 +02:00
netxlite fix(netxlite): robust {ReadAll,Copy}Context with wrapped io.EOF (#661) 2022-01-12 14:26:10 +01:00
ooapi feat(httpx): improve testing using the fakefiller (#649) 2022-01-05 14:49:31 +01:00
platform fix(internal/platform): support freebsd (#445) 2021-08-20 14:00:06 +02:00
ptx cleanup: merge legacy errorsx in netxlite and hide classifiers (#655) 2022-01-07 17:31:21 +01:00
randx doc: improve and reference existing bug in the code (#356) 2021-06-04 12:50:23 +02:00
runtimex refactor(tor): rewrite using measurex (#652) 2022-01-05 18:41:11 +01:00
scrubber refactor: interfaces and data types into the model package (#642) 2022-01-03 13:53:23 +01:00
shellx refactor: interfaces and data types into the model package (#642) 2022-01-03 13:53:23 +01:00
stuninput refactor: create common package for holding STUN input (#631) 2021-12-03 14:45:25 +01:00
tunnel refactor: interfaces and data types into the model package (#642) 2022-01-03 13:53:23 +01:00
tutorial refactor: interfaces and data types into the model package (#642) 2022-01-03 13:53:23 +01:00
version chore: set master's version to 3.14.0-alpha (#638) 2021-12-06 17:54:58 +01:00
README.md refactor: interfaces and data types into the model package (#642) 2022-01-03 13:53:23 +01:00

Directory github.com/ooni/probe-cli/internal

This directory contains private Go packages.

As a reminder, you can always check the Go documentation of a package by using

go doc -all ./internal/$package

where $package is the name of the package.

Some notable packages:

  • model contains the interfaces and data model shared by most packages inside this directory;

  • netxlite is the underlying networking library;

  • tutorial contains tutorials on writing new experiments, using measurements libraries, and networking code.