ooni-probe-cli/internal
Simone Basso 16f7407b13
feat(netxlite): observe additional DNS-over-UDP responses (#762)
This diff introduces support for observing additional DNS-over-UDP
responses in some censored environments (e.g. China).

After some uncertainty around whether to use connected or unconnected
UDP sockets, I eventually settled for connected.

Here's a recap:

|                         | connected | unconnected |
| ----------------------- | --------- | ----------- |
| see ICMP errors         | ✔️         |           |
| responses from any server |         | ✔️           |

Because most if not all DNS resolvers expect answers from exactly
the same servers to which they sent the query, I would say that
it's more important to have some limited ability of observing the
effect of ICMP errors (e.g., host_unreachable when we set a low
TTL and send out a query to a server).

Therefore, my choice was to modify the existing DNS-over-UDP transport.

Here's an overview of the changes:

1. introduce a new API for performing an async round trip that returns
a channel wrapper where all responses are posted. The channel will not ever
be closed, so the reader needs to use select for safely reading. If the
reader users the wrapper's Next or TryNextResponses methods, these details
do not matter because they already implement a safe reading pattern.

2. the async round trip API performs the round trip in the background
and stops processing when it sees the first error.

3. the background running code will use an overall deadline derived
from the DNSTransport.IOTimeout field to know when to stop.

4. the background running code will additionally stop running if
noone is reading the channel and there are no empty slots in the
channel's buffer.

5. the RoundTrip method has been rewritten in terms of the async API.

The design I'm using here implements the proposal for async round
trips defined at https://github.com/ooni/probe/issues/2099. I have
chosen not to make all transports async because the DNS transport
seems the only transport that needs to also work in async mode.

While there, I noticed that we were not propagating CloseIdleConnection
to the underlying dialer, which was potentially wrong, so I did it.
2022-05-26 20:09:00 +02:00
..
atomicx doc: cleanup and improve for recently moved pkgs (#354) 2021-06-04 11:39:00 +02:00
bytecounter feat(torsf): collect tor logs, select rendezvous method, count bytes (#683) 2022-02-07 17:05:36 +01:00
cmd Spring cleanup: remove unused/unneded code (#761) 2022-05-25 13:21:39 +02:00
engine refactor: DNSTransport I/Os DNS messages (#760) 2022-05-25 17:03:58 +02:00
fakefill feat(httpx): improve testing using the fakefiller (#649) 2022-01-05 14:49:31 +01:00
fsx refactor: merge dnsx and errorsx into netxlite (#517) 2021-09-28 12:42:01 +02:00
httpx fix(httpx): correctly combine paths (#706) 2022-05-09 21:32:49 +02:00
humanize fix(all): introduce and use iox.CopyContext (#380) 2021-06-15 13:44:28 +02:00
kvstore refactor: interfaces and data types into the model package (#642) 2022-01-03 13:53:23 +01:00
legacy/assetsdir cleanup: move legacy from internal/engine to internal (#759) 2022-05-25 10:19:03 +02:00
measurex refactor: DNSTransport I/Os DNS messages (#760) 2022-05-25 17:03:58 +02:00
mlablocate cleanup: remove redundant HTTPClient definition (#643) 2022-01-03 16:47:54 +01:00
mlablocatev2 cleanup: remove redundant HTTPClient definition (#643) 2022-01-03 16:47:54 +01:00
model refactor: DNSTransport I/Os DNS messages (#760) 2022-05-25 17:03:58 +02:00
multierror doc: cleanup and improve for recently moved pkgs (#354) 2021-06-04 11:39:00 +02:00
netxlite feat(netxlite): observe additional DNS-over-UDP responses (#762) 2022-05-26 20:09:00 +02:00
platform feat: add support for OpenBSD (#703) 2022-03-08 12:25:33 +01:00
ptx feat(torsf): collect tor logs, select rendezvous method, count bytes (#683) 2022-02-07 17:05:36 +01:00
randx doc: improve and reference existing bug in the code (#356) 2021-06-04 12:50:23 +02:00
runtimex refactor(tor): rewrite using measurex (#652) 2022-01-05 18:41:11 +01:00
scrubber refactor: interfaces and data types into the model package (#642) 2022-01-03 13:53:23 +01:00
shellx refactor: interfaces and data types into the model package (#642) 2022-01-03 13:53:23 +01:00
stuninput refactor: create common package for holding STUN input (#631) 2021-12-03 14:45:25 +01:00
torlogs feat: re-implement the vanilla_tor experiment (#718) 2022-05-10 15:43:28 +02:00
tunnel feat(torsf): collect tor logs, select rendezvous method, count bytes (#683) 2022-02-07 17:05:36 +01:00
tutorial cli: upgrade to lucas-clemente/quic-go@v0.27.0 (#715) 2022-05-06 12:24:03 +02:00
version chore: we're not hacking on v3.16.0-alpha (#749) 2022-05-20 20:03:48 +02:00
README.md refactor: interfaces and data types into the model package (#642) 2022-01-03 13:53:23 +01:00

Directory github.com/ooni/probe-cli/internal

This directory contains private Go packages.

As a reminder, you can always check the Go documentation of a package by using

go doc -all ./internal/$package

where $package is the name of the package.

Some notable packages:

  • model contains the interfaces and data model shared by most packages inside this directory;

  • netxlite is the underlying networking library;

  • tutorial contains tutorials on writing new experiments, using measurements libraries, and networking code.