feat(cli/run): read inputs from file/cmdline (#180)

This diff introduces the possibility of specifying --input-file file
multiple times to force ooniprobe to read inputs from file.

Like we do for miniooni, the file shall contain a single entry per
line and this entry should be a URL for websites.

Likewise, one can specify --input URL multiple times.

This implementation is a very simple, initial implementation and there
is a bunch of changes I'd like to add on top of it.

And also perhaps a bunch of cleanups.

I've chosen to expose these flag _only_ for websites for now.

Part of https://github.com/ooni/probe/issues/1283.
This commit is contained in:
Simone Basso 2020-12-01 07:21:33 +01:00 committed by GitHub
commit cfd3b8f9b2
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
3 changed files with 59 additions and 19 deletions

View file

@ -44,6 +44,15 @@ type Controller struct {
msmts map[int64]*database.Measurement
inputIdxMap map[int64]int64 // Used to map mk idx to database id
// InputFiles optionally contains the names of the input
// files to read inputs from (only for nettests that take
// inputs, of course)
InputFiles []string
// Inputs contains inputs to be tested. These are specified
// using the command line using the --input flag.
Inputs []string
// numInputs is the total number of inputs
numInputs int

View file

@ -12,6 +12,8 @@ func lookupURLs(ctl *Controller, limit int64, categories []string) ([]string, ma
inputloader := engine.NewInputLoader(engine.InputLoaderConfig{
InputPolicy: engine.InputRequired,
Session: ctl.Session,
SourceFiles: ctl.InputFiles,
StaticInputs: ctl.Inputs,
URLCategories: categories,
URLLimit: limit,
})