* fix(model/archival.go): more optional keys
Basically, `t0` and `transaction_id` should be optional. Version 0.4.x
of web_connectivity should not include them, version 0.5.x should.
There is a technical reason why v0.4.x should not include them. The code
it is based on, tracex, does not record these two fields.
Whereas, v0.5.x, uses measurexlite, which records these two fields.
Part of https://github.com/ooni/probe/issues/2238
* fix(webconnectivity@v0.5): add more fields
This diff adds the following fields to webconnectivity@v0.5:
1. agent, always set to "redirect" (legacy field);
2. client_resolver, properly initialized w/ the resolver's IPv4 address;
3. retries, legacy field always set to null;
4. socksproxy, legacy field always set to null.
Part of https://github.com/ooni/probe/issues/2238
* fix(webconnectivity@v0.5): register extensions
The general idea behind this field is that we would be able
in the future to tweak the data model for some fields, by declaring
we're using a later version, so it seems useful to add it.
See https://github.com/ooni/probe/issues/2238
* fix(measurexlite): use tcp or quic for tls handshake network
This diff fixes a bug where measurexlite was using "tls" as the
protocol for the TLS handshake when using TCP.
While this choice _could_ make sense, the rest of the code we have
written so far uses "tcp" instead.
Using "tcp" makes more sense because it allows you to search for
the same endpoint across different events by checking for the same
network and for the same endpoint rather than special casing TLS
handshakes for using "tls" when the endpoint is "tcp".
See https://github.com/ooni/probe/issues/2238
* chore: run alltests.yml for "alltestsbuild" branches
Part of https://github.com/ooni/probe/issues/2238
This diff modifies webconnectivity@v0.5 to take decisions regarding
TLS blocking by using the response from the TH rather than using
questionable heuristics based on inspecting the TLSHandshake list
alone. This change should improve correctness _when_ we're using
the improved TH, which is currently used for 50% of the probes.
See https://github.com/ooni/probe/issues/2257
While there, modify `control.go` to specify which control is being used.
A bunch of packages (including oohelperd) just need the ability to
use MaxMind-like databases. They don't need the additional functionality
implemented by the geolocate package. Such a package, in fact, is
mostly (if not only) needed by the engine package.
Therefore, move code to query MaxMind-like databases to a separate
package, and avoid depending on geolocate in all the packages for
which it's sufficient to use geoipx.
Part of https://github.com/ooni/probe/issues/2240
See what we documented at https://github.com/ooni/spec/pull/257
Reference issue: https://github.com/ooni/probe/issues/2238
See also the related ooni/spec PR: https://github.com/ooni/spec/pull/257
See also https://github.com/ooni/probe/issues/2237
While there, bump webconnectivity@v0.5 version because this change
has an impact onto the generated data format.
The drop in coverage is unavoidable because we've written some
tests for `measurex` to ensure we deal with DNS resolvers and transport
names correctly depending on the splitting policy we use.
(However, `measurex` is only used for the `tor` experiment and, per
the step-by-step design document, new experiments should use
`measurexlite` instead, so this is hopefully fine(TM).)
While there, fix a broken integration test that does not run in `-short` mode.
