Commit Graph

10 Commits

Author SHA1 Message Date
Simone Basso
beba543b98
cleanup: use NewHTTPTransportWithResolver more often (#808)
We can simplify code in a bunch of places using a useful factory.

Part of https://github.com/ooni/probe/issues/2121.
2022-06-08 15:13:01 +02:00
Simone Basso
dea23b49d5
refactor(oohelperd): use netxlite rather than netx (#806)
The oohelperd implementation did not actually need using netx because
it was just constructing default types with logging, which is what
netxlite already does. Hence, let's avoid using netx here.

See https://github.com/ooni/probe/issues/2121
2022-06-08 10:14:10 +02:00
Simone Basso
7a0a156aec
Spring cleanup: remove unused/unneded code (#761)
* cleanup: remove the archival package

See https://github.com/ooni/probe/issues/2116

* cleanup: remove websteps fall 2021 edition

See https://github.com/ooni/probe/issues/2116

* cleanup: remove JavaScript based testing framework

https://github.com/ooni/probe/issues/2116

* cleanup: remove the unused ooapi package

See https://github.com/ooni/probe/issues/2116
2022-05-25 13:21:39 +02:00
Simone Basso
1776ea1288
cleanup: remove websteps summer 2021 implementation (#722)
See https://github.com/ooni/probe/issues/2094
2022-05-13 15:06:03 +02:00
Simone Basso
566c6b246a
cleanup: remove unnecessary legacy interfaces (#656)
This diff addresses another point of https://github.com/ooni/probe/issues/1956:

> - [ ] observe that we're still using a bunch of private interfaces for common interfaces such as the `Dialer`, so we can get rid of these private interfaces and always use the ones in `model`, which allows us to remove a bunch of legacy wrappers

Additional cleanups may still be possible. The more I cleanup, the more I see
there's extra legacy code we can dispose of (which seems good?).
2022-01-07 18:33:37 +01:00
Simone Basso
ba9151d4fa
feat(webstepsx): websteps using measurex (#530)
This diff adds the prototype websteps implementation that used
to live at https://github.com/ooni/probe-cli/pull/506.

The code is reasonably good already and it's pointing to a roaming
test helper that I've properly configured.

You can run websteps with:

```
./miniooni -n websteps
```

This will go over the test list for your country.

At this stage the mechanics of the experiment is set, but we
still need to have a conversation on the following topics:

1. whether we're okay with reusing the data format used by other
OONI experiments, or we would like to use a more compact data
format (which may either be a more compact JSON or we can choose
to always submit compressed measurements for websteps);

2. the extent to which we would like to keep the measurement as
a collection of "the experiment saw this" and "the test helper
saw that" and let the pipeline choose an overall score: this is
clearly an option, but there is also the opposite option to
build a summary of the measurement on the probe.

Compared to the previous prototype of websteps, the main
architectural change we have here is that we are following
the point of view of the probe and the test helper is
much more dumb. Basically, the probe will choose which
redirection to follow and ask the test helper every time
it discovers a new URL to measure it w/o redirections.

Reference issue: https://github.com/ooni/probe/issues/1733
2021-09-30 02:06:27 +02:00
Simone Basso
985c1ba761
fix(oohelperd): reduce differences with legacy helper (#504)
Part of https://github.com/ooni/probe/issues/1707
2021-09-27 08:13:30 +02:00
Simone Basso
ce854e8ae1
refactor(oohelperd): better distinguish different helpers (#434)
Part of https://github.com/ooni/probe/issues/1733
2021-08-17 11:23:53 +02:00
kelmenhorst
c31591f298
cli: new testhelper and the websteps experiment prototype (#432)
This is the extension of https://github.com/ooni/probe-cli/pull/431, and my final deliverable for GSoC 2021.

The diff introduces:

1) The new `testhelper` which supports testing multiple IP endpoints per domain and introduces HTTP/3 control measurements. The specification of the `testhelper` can be found at https://github.com/ooni/spec/pull/219. The `testhelper` algorithm consists of three main steps:

   * `InitialChecks` verifies that the input URL can be parsed, has an expected scheme, and contains a valid domain name.

   * `Explore` enumerates all the URLs that it discovers by redirection from the original URL, or by detecting h3 support at the target host.

   * `Generate` performs a step-by-step measurement of each discovered URL.

2) A prototype of the corresponding new experiment `websteps` which uses the control measurement of the `testhelper` to know which URLs to measure, and what to expect. The prototype does not yet have:

   * unit and integration tests,

   * an analysis tool to compare the control and the probe measurement.

This PR is my final deliverable as it is the outcome of the trials, considerations and efforts of my GSoC weeks at OONI. 
It fully integrates HTTP/3 (QUIC) support which has been only used in the `urlgetter` experiment until now.

Related issues: https://github.com/ooni/probe/issues/1729 and https://github.com/ooni/probe/issues/1733.
2021-08-17 10:29:06 +02:00
Simone Basso
4eeadd06a5
refactor: move more commands to internal/cmd (#207)
* refactor: move more commands to internal/cmd

Part of https://github.com/ooni/probe/issues/1335.

We would like all commands to be at the same level of engine
rather than inside engine (now that we can do it).

* fix: update .gitignore

* refactor: also move jafar outside engine

* We should be good now?
2021-02-03 12:23:15 +01:00