ooni-probe-cli/ooni.go
Simone Basso b9b555ba68 Integrate further with ooni/probe-engine: episode two (#46)
* utils/geoip.go: use github.com/ooni/probe-engine

Let's start using the engine by rewriting utils/geoip.go to
be just a thin wrapper around the engine functionality.

* Ready for review

* Checkpoint: the im tests are converted

Still have some doubts with respect to the variables that
are passed to MK via probe-engine. Will double check.

* fix(i/c/r/run.go): write the correct logic

* nettests: one more comment and also fix a format string

* Tweak previous

* progress

* Fix doofus

* better comment

* XXX => actionable comment

* Add glue to simplify test keys management

Making the concept of measurement more abstract in the engine is
not feasible because, when submitting a measurement, we need to
modify it to update the report ID and the measurement ID. Therefore,
returning a serialized measurement is not a good idea. We will
keep using a model.Measurement in the engine.

Changing model.Measurement.TestKeys's type from a `interface{}`
pointing to a well defined data structure to `map[string]interface{}`
is a regression because means that we are moving from code that
has a clear and defined structure to code that is more complicated
to parse and validate. Since we're already suffering havily from
the lack of a good schema, I'm not going to make the situation
worst by worsening the engine. At least for ndt7 and psiphon, we
now have a good schema and I don't want to lose that.

However, the current code in this repository is expecting the
test keys to be a `map[string]interface{}`. This choice was
dictated by the fact that we receive a JSON from Measurement Kit
and by the fact that there's not a clear schema.

To solve this tension, in this commit I am going to write glue
adapter code that makes sure that the TestKeys of a Measurement
are converted to `map[string]interface{}`. This will be done
using a type cast where possible and JSON serialization and parsing
otherwise. In a perfect world, glue is not a good idea, but in a
real world it may actually be useful.

When all tests in the engine will have a clear Go data structure,
we'll then remove the glue and just cast to the proper data
structure from `interface{}` where required.

* nettests/performance: use probe-engine

* go.{mod,sum}: upgrade to latest probe-engine

* nettests/middlebox: use ooni/probe-engine

* Update to the latest probe-engine

* web_connectivity: rewrite to use probe-engine

* Cosmetic change suggested by @hellais

* nettests/nettests.go: remove unused code

* nettests/nettests.go: fix progress

* nettests/nettests.go: remove go-measurement-kit code

* We don't depend on go-measurement-kit anymore

* Improve non-verbose output where possible

See also: https://github.com/measurement-kit/measurement-kit/issues/1856

* Make web_connectivity output pleasant

* Update to the latest probe-engine

* nettests/nettests.go: honour sharing settings

* Update to the latest probe-engine

* Use log.WithFields for probe-engine

* Update go.mod go.sum

* Revert "Update go.mod go.sum"

This reverts commit 5ecd38d8236f4a4e9b77ddb8e8a0d1e3cdd4b818.

* Revert "Revert "Update go.mod go.sum""

This reverts commit 6114b31eca98826112032776bd0feff02d763ecd.

* Upgrade ooni/probe-engine

* Unset GOPATH before running go build commands

* Dockefile: fix linux build by using latest

* Update to the latest ooni/probe-engine

```
go get -u github.com/ooni/probe-engine
go mod tidy
```

* Repair build
2019-08-15 18:08:43 +02:00

157 lines
3.5 KiB
Go

package ooni
import (
"context"
"io/ioutil"
"os"
"github.com/apex/log"
"github.com/ooni/probe-cli/config"
"github.com/ooni/probe-cli/internal/bindata"
"github.com/ooni/probe-cli/internal/database"
"github.com/ooni/probe-cli/internal/enginex"
"github.com/ooni/probe-cli/internal/legacy"
"github.com/ooni/probe-cli/internal/onboard"
"github.com/ooni/probe-cli/utils"
"github.com/ooni/probe-cli/version"
"github.com/ooni/probe-engine/session"
"github.com/pkg/errors"
"upper.io/db.v3/lib/sqlbuilder"
)
// Context for OONI Probe
type Context struct {
Config *config.Config
DB sqlbuilder.Database
IsBatch bool
Session *session.Session
Home string
TempDir string
dbPath string
configPath string
}
// MaybeLocationLookup will lookup the location of the user unless it's already cached
func (c *Context) MaybeLocationLookup() error {
return c.Session.MaybeLookupLocation(context.Background())
}
// MaybeOnboarding will run the onboarding process only if the informed consent
// config option is set to false
func (c *Context) MaybeOnboarding() error {
if c.Config.InformedConsent == false {
if c.IsBatch == true {
return errors.New("cannot run onboarding in batch mode")
}
if err := onboard.Onboarding(c.Config); err != nil {
return errors.Wrap(err, "onboarding")
}
}
return nil
}
// Init the OONI manager
func (c *Context) Init() error {
var err error
if err = legacy.MaybeMigrateHome(); err != nil {
return errors.Wrap(err, "migrating home")
}
if err = MaybeInitializeHome(c.Home); err != nil {
return err
}
if c.configPath != "" {
log.Debugf("Reading config file from %s", c.configPath)
c.Config, err = config.ReadConfig(c.configPath)
} else {
log.Debug("Reading default config file")
c.Config, err = InitDefaultConfig(c.Home)
}
if err != nil {
return err
}
c.dbPath = utils.DBDir(c.Home, "main")
log.Debugf("Connecting to database sqlite3://%s", c.dbPath)
db, err := database.Connect(c.dbPath)
if err != nil {
return err
}
c.DB = db
tempDir, err := ioutil.TempDir("", "ooni")
if err != nil {
return errors.Wrap(err, "creating TempDir")
}
c.TempDir = tempDir
return nil
}
// NewContext creates a new context instance.
func NewContext(configPath string, homePath string) *Context {
return &Context{
Home: homePath,
Config: &config.Config{},
configPath: configPath,
Session: session.New(
enginex.Logger,
"ooniprobe-desktop",
version.Version,
utils.AssetsDir(homePath),
nil, // explicit proxy url.URL
nil, // explicit tls.Config
),
}
}
// MaybeInitializeHome does the setup for a new OONI Home
func MaybeInitializeHome(home string) error {
for _, d := range utils.RequiredDirs(home) {
if _, e := os.Stat(d); e != nil {
if err := os.MkdirAll(d, 0700); err != nil {
return err
}
}
}
return nil
}
// InitDefaultConfig reads the config from common locations or creates it if
// missing.
func InitDefaultConfig(home string) (*config.Config, error) {
var (
err error
c *config.Config
configPath = utils.ConfigPath(home)
)
c, err = config.ReadConfig(configPath)
if err != nil {
if os.IsNotExist(err) {
log.Debugf("writing default config to %s", configPath)
var data []byte
data, err = bindata.Asset("data/default-config.json")
if err != nil {
return nil, err
}
err = ioutil.WriteFile(
configPath,
data,
0644,
)
if err != nil {
return nil, err
}
return InitDefaultConfig(home)
}
return nil, err
}
return c, nil
}