16f7407b13
This diff introduces support for observing additional DNS-over-UDP responses in some censored environments (e.g. China). After some uncertainty around whether to use connected or unconnected UDP sockets, I eventually settled for connected. Here's a recap: | | connected | unconnected | | ----------------------- | --------- | ----------- | | see ICMP errors | ✔️ | ❌ | | responses from any server | ❌ | ✔️ | Because most if not all DNS resolvers expect answers from exactly the same servers to which they sent the query, I would say that it's more important to have some limited ability of observing the effect of ICMP errors (e.g., host_unreachable when we set a low TTL and send out a query to a server). Therefore, my choice was to modify the existing DNS-over-UDP transport. Here's an overview of the changes: 1. introduce a new API for performing an async round trip that returns a channel wrapper where all responses are posted. The channel will not ever be closed, so the reader needs to use select for safely reading. If the reader users the wrapper's Next or TryNextResponses methods, these details do not matter because they already implement a safe reading pattern. 2. the async round trip API performs the round trip in the background and stops processing when it sees the first error. 3. the background running code will use an overall deadline derived from the DNSTransport.IOTimeout field to know when to stop. 4. the background running code will additionally stop running if noone is reading the channel and there are no empty slots in the channel's buffer. 5. the RoundTrip method has been rewritten in terms of the async API. The design I'm using here implements the proposal for async round trips defined at https://github.com/ooni/probe/issues/2099. I have chosen not to make all transports async because the DNS transport seems the only transport that needs to also work in async mode. While there, I noticed that we were not propagating CloseIdleConnection to the underlying dialer, which was potentially wrong, so I did it. |
||
|---|---|---|
| .github/workflows | ||
| .vscode | ||
| CLI | ||
| cmd | ||
| docs | ||
| E2E | ||
| internal | ||
| MOBILE | ||
| pkg | ||
| QA | ||
| script | ||
| testdata | ||
| .eslintrc.json | ||
| .gitignore | ||
| CODE_OF_CONDUCT.md | ||
| CODEOWNERS | ||
| CONTRIBUTING.md | ||
| go.mod | ||
| go.sum | ||
| LICENSE | ||
| mk | ||
| NOTICE.md | ||
| PULL_REQUEST_TEMPLATE.md | ||
| Readme.md | ||
| testjafar.bash | ||
OONI Probe Client Library and CLI
The Open Observatory of Network Interference (OONI) is a non-profit free software project that aims to empower decentralized efforts in documenting Internet censorship around the world.
This repository contains core OONI tools written in Go:
-
the CLI client (cmd/ooniprobe);
-
the test helper server (internal/cmd/oohelperd);
-
the mobile library (pkg/oonimkall);
-
and all the related support packages (inside internal).
Every top-level directory in this repository contains an explanatory README file. You
may also notice that some internal packages live under internal/engine
while most others are top-level. This is part of a long-standing refactoring started when we merged
https://github.com/ooni/probe-engine into this repository. We'll slowly
ensure that all packages inside engine are moved out of it and inside internal.
Semantic versioning policy
The mobile library is a public package for technical reasons. Go mobile tools require
a public package to build from. Yet, we don't consider API breakages happening in
such a package to be sufficient to bump our major version number. For us, the mobile
library is just a mean to implement OONI Probe Android and OONI Probe iOS. We'll
only bump the major version number if we change ./cmd/ooniprobe's CLI.
License
SPDX-License-Identifier: GPL-3.0-or-later
User setup
Please, follow the instructions at ooni.org/install/cli
to install ooniprobe. If we do not support your use case, please let us know. Once
ooniprobe is installed, try ooniprobe help to get interactive help.
Reporting issues
Report issues at github.com/ooni/probe.
Please, make sure you add the ooni/probe-cli label.
Build instructions
ooniprobe
Be sure you have golang 1.18.2 and a C compiler (Mingw-w64 for Windows). You can build using:
go build -v ./cmd/ooniprobe
This will generate a binary called ooniprobe in the current directory.
Android bindings
Make sure you have GNU make installed, then run:
./mk ./MOBILE/android
to build bindings for Android. (Add OONI_PSIPHON_TAGS="" if you
cannot clone private repositories in the https://github.com/ooni namespace.)
The generated bindings are (manually) pushed to the Maven Central package repository. The instructions explaining how to integrate these bindings are published along with the release notes.
iOS bindings
Make sure you have GNU make installed, then run:
./mk ./MOBILE/ios
to build bindings for iOS. (Add OONI_PSIPHON_TAGS="" if you
cannot clone private repositories in the https://github.com/ooni namespace.)
The generated bindings are (manually) added to GitHub releases. The instructions explaining how to integrate these bindings are published along with the release notes.
miniooni
Miniooni is the experimental OONI client used for research. Compile using:
go build -v ./internal/cmd/miniooni
This will generate a binary called miniooni in the current directory.
oohelperd
Oohelperd is the test helper server. Compile using:
go build -v ./internal/cmd/oohelperd
This will generate a binary called oohelperd in the current directory.
Specifications
Every nettest (aka experiment) implemented in this repository has a companion spec in ooni/spec.
Contributing
Please, see CONTRIBUTING.md.
Updating dependencies
go get -u -v -d ./... && go mod tidy
Releasing
Create an issue according to the routine release template and perform any item inside the check-list.
We build releases using ./mk, which requires GNU make. Try
the ./mk help|less command for detailed usage.