Commit Graph

37 Commits

Author SHA1 Message Date
Arturo Filastò
58199a020e Refactoring to reduce package count
* Consolidate util and utils into the same package
* Move internal/onboard into internal/cli/onboard
* Move maybeOnboard into the onboard package
2019-12-02 15:15:50 +02:00
Arturo Filastò
8a27de6c7d
Remove all settings which are not implemented (#73)
* Remove all settings which are not implemented

* Add support for migrating config files from version 0 -> 1

Add basic unittests for the config file migration

* Update the config file used by travis

* Fix typos in console log messages

* Improve the comment

* Fix type of limit

* Set informed consent to true in the travis config
2019-11-14 17:58:31 +01:00
Simone Basso
946289d6fe
Use more general ooni/probe-engine API (#67)
No functional change, just importing less stuff and meddling
much less with the internals of ooni/probe-engine.
2019-10-28 14:05:05 +01:00
Simone Basso
b9b555ba68 Integrate further with ooni/probe-engine: episode two (#46)
* utils/geoip.go: use github.com/ooni/probe-engine

Let's start using the engine by rewriting utils/geoip.go to
be just a thin wrapper around the engine functionality.

* Ready for review

* Checkpoint: the im tests are converted

Still have some doubts with respect to the variables that
are passed to MK via probe-engine. Will double check.

* fix(i/c/r/run.go): write the correct logic

* nettests: one more comment and also fix a format string

* Tweak previous

* progress

* Fix doofus

* better comment

* XXX => actionable comment

* Add glue to simplify test keys management

Making the concept of measurement more abstract in the engine is
not feasible because, when submitting a measurement, we need to
modify it to update the report ID and the measurement ID. Therefore,
returning a serialized measurement is not a good idea. We will
keep using a model.Measurement in the engine.

Changing model.Measurement.TestKeys's type from a `interface{}`
pointing to a well defined data structure to `map[string]interface{}`
is a regression because means that we are moving from code that
has a clear and defined structure to code that is more complicated
to parse and validate. Since we're already suffering havily from
the lack of a good schema, I'm not going to make the situation
worst by worsening the engine. At least for ndt7 and psiphon, we
now have a good schema and I don't want to lose that.

However, the current code in this repository is expecting the
test keys to be a `map[string]interface{}`. This choice was
dictated by the fact that we receive a JSON from Measurement Kit
and by the fact that there's not a clear schema.

To solve this tension, in this commit I am going to write glue
adapter code that makes sure that the TestKeys of a Measurement
are converted to `map[string]interface{}`. This will be done
using a type cast where possible and JSON serialization and parsing
otherwise. In a perfect world, glue is not a good idea, but in a
real world it may actually be useful.

When all tests in the engine will have a clear Go data structure,
we'll then remove the glue and just cast to the proper data
structure from `interface{}` where required.

* nettests/performance: use probe-engine

* go.{mod,sum}: upgrade to latest probe-engine

* nettests/middlebox: use ooni/probe-engine

* Update to the latest probe-engine

* web_connectivity: rewrite to use probe-engine

* Cosmetic change suggested by @hellais

* nettests/nettests.go: remove unused code

* nettests/nettests.go: fix progress

* nettests/nettests.go: remove go-measurement-kit code

* We don't depend on go-measurement-kit anymore

* Improve non-verbose output where possible

See also: https://github.com/measurement-kit/measurement-kit/issues/1856

* Make web_connectivity output pleasant

* Update to the latest probe-engine

* nettests/nettests.go: honour sharing settings

* Update to the latest probe-engine

* Use log.WithFields for probe-engine

* Update go.mod go.sum

* Revert "Update go.mod go.sum"

This reverts commit 5ecd38d8236f4a4e9b77ddb8e8a0d1e3cdd4b818.

* Revert "Revert "Update go.mod go.sum""

This reverts commit 6114b31eca98826112032776bd0feff02d763ecd.

* Upgrade ooni/probe-engine

* Unset GOPATH before running go build commands

* Dockefile: fix linux build by using latest

* Update to the latest ooni/probe-engine

```
go get -u github.com/ooni/probe-engine
go mod tidy
```

* Repair build
2019-08-15 18:08:43 +02:00
Simone Basso
df629237be
Use ooni/probe-engine: episode 1 (#42) 2019-05-23 16:38:46 +02:00
Arturo Filastò
2cda2d9639 Bump version 2019-05-15 19:26:24 +02:00
Arturo Filastò
1490d07089 Bump version number 2019-05-15 14:57:02 +02:00
Simone Basso
9992690f8f Use MK v0.10.x, zap legacy GeoIP, build also on Linux (#29)
* Gopkg.lock: use MK v0.10.3

* ooni: stop using legacy GeoIP database files

* Some yak shaving of Makefile

1. remove now broken commands to download deps

2. also define the CXX cross compiler

* chore(dep): migrate from dep to go 1.11+ modules

See https://blog.callr.tech/migrating-from-dep-to-go-1.11-modules/

I need this to simplify my life in building for Travis.

* Introduce build.sh and repair build

In going forward, I believe we don't actually need a Makefile but I
didn't want to make such a radical change now.

* Another strategy wrt gopath

* travis: run regress tests on macOS

Closes #30
2019-05-15 15:48:06 +03:00
Arturo Filastò
ec337e8186 Bump to beta.1 2018-09-27 18:39:35 +02:00
Arturo Filastò
1b90b6f1ec Bump to alpha.3 2018-09-27 11:43:13 +02:00
Arturo Filastò
5567fe1d2b Bump version number of probe-cli 2018-09-25 14:21:18 +02:00
Arturo Filastò
1fe8eedfb3 Bump ooni version number 2018-09-17 16:38:41 +02:00
Arturo Filastò
d55cccc236 Exit with non-zero code when trying to do interactive onboarding and --batch is set 2018-09-17 16:16:56 +02:00
Arturo Filastò
ff2f973523 Start integrating upper/db as a new ORM 2018-09-05 18:40:37 +02:00
Arturo Filastò
45aea93e6b Fix default config and version 2018-07-14 16:58:08 +02:00
Arturo Filastò
78cf8d6ca2 Create config file when it's missing 2018-06-25 17:49:17 +02:00
Arturo Filastò
a6b95f50c9 Move onboard into a separate package 2018-06-25 17:14:29 +02:00
Arturo Filastò
15c901ed68 Make the download of geoip data files conditional 2018-06-22 11:17:51 +02:00
Arturo Filastò
1bba0c7899 Add basic unittests for the config related functionality 2018-06-22 11:01:15 +02:00
Arturo Filastò
2653a3f67f Consolidate config related functionality into the config package 2018-06-22 10:29:47 +02:00
Arturo Filastò
c620bc9726 Make path and homedir related logic more robust
Add ability to pass OONI_HOME environment variable
2018-05-21 17:33:59 -07:00
Arturo Filastò
0c5b6aa37c Rename gooni to probe-cli
This is the command I used:
```
rg 'github.com/openobservatory/gooni' --files-with-matches \
    | xargs sed -i '' 's/github.com\/openobservatory\/gooni/github.com\/ooni\/probe-cli/g'
```
2018-05-03 14:59:55 +02:00
Arturo Filastò
cffb9ea74c Move all directory related functionality into paths utils
As per suggestion by @willscott in: https://github.com/OpenObservatory/gooni/pull/9#discussion_r176760264
2018-03-27 15:09:34 +02:00
Arturo Filastò
9fe917f294 On first run also download the geoip data files 2018-03-23 15:38:25 +01:00
Arturo Filastò
00859e87a6 Get web_connectivity to work with dynamic URL fetching 2018-03-23 12:41:06 +01:00
Arturo Filastò
7fe1551951 Refactor how we create the context 2018-03-23 12:10:14 +01:00
Arturo Filastò
90c1c2de87 Start adding support for writing measurements to disk 2018-03-19 13:20:42 +01:00
Arturo Filastò
efb7e87d1b Refactor the EnsureDefaultConfigDir() function 2018-03-19 11:23:54 +01:00
Will Scott
62b515e2e0 pull out directory creation to new function 2018-03-08 11:46:31 -08:00
Will Scott
90edffb1d7 ensure presence of home and home/db dir's during creation of the default config file 2018-03-08 05:07:25 -08:00
Will Scott
7151f4270f config default path / writing / onboarding occurs 2018-03-08 02:33:27 -08:00
Will Scott
8d5da14c4b Initial creation of the '.ooni/db' folders when not yet setup 2018-03-08 02:25:40 -08:00
Arturo Filastò
d3d3ce9d78 Get NDT test to run via gooni 2018-02-13 17:16:23 +02:00
Arturo Filastò
ca231bca9f Start fleshing out database related functions 2018-02-12 18:29:03 +02:00
Arturo Filastò
6008b5c7c5 Config file workflow management 2018-02-12 17:45:13 +02:00
Arturo Filastò
602a5dc952 Add functions to handling migration from legacy OONI Probe 2018-02-12 12:18:14 +02:00
Arturo Filastò
7327e1ff7f Start laying out the structure of gooni 2018-02-07 20:02:18 +02:00