1. reduce the number of periodic builds
2. just build as part of the release process in most cases
3. shorttests duplicates coverage
Preliminary changes as part of https://github.com/ooni/probe/issues/1466
Just consistency. There was no clear semantic regarding why
sometimes I was using __name and sometimes _name.
So prefer _name everywhere.
Occurred to me while working on https://github.com/ooni/probe/issues/1466.
After all the refactoring done so far, we can run checks directly
inside of `make`, because we have auto-cleanup, temporary environments
and we don't need wrapper scripts anymore.
Part of https://github.com/ooni/probe/issues/1466.
The previous fix allowed us to return values directly from
implementations of Engine. Here we take advantage of this
possibility by immediately refactoring how backticks work.
Part of https://github.com/ooni/probe/issues/1466.
The previous composition strategy was such that it was difficult
to compose functions returning a value.
This new composition strategy is better because it allows us to
return values, which is definitely going to help us.
Part of https://github.com/ooni/probe/issues/1466.
* fix(tunnel): pass /absolute/path/to/tor to cretz/bine
It seems cretz/bine is not aware of https://blog.golang.org/path-security
for now. I am planning to send over a diff for that later today.
In the meanwhile, do the right thing here, and make sure that we obtain
the absolute path to the tor binary before we continue.
This work is part of https://github.com/ooni/probe-engine/issues/283.
* fix tests when tor is not installed
* Add support for tracking the is_uploaded status in the results table
* Add unit tests for measurement upload status
This implements: https://github.com/ooni/probe/issues/1457
* Update cmd/ooniprobe/internal/database/actions.go
Co-authored-by: Simone Basso <bassosimone@gmail.com>
* fix(make): correctly write oonimkall.podspec
Part of https://github.com/ooni/probe/issues/1439
* chore: set version number to v3.10.0-beta.1
* fix(ios): don't build a target that requires git
This commit merges the mobile-staging branch into the main history so that
we are not losing track of what it happened during its lifetime.
We're going to not need this branch anymore, because we're now going to
build releases on a special purpose build machine.
At the same time, we don't want to loose track of the history of such a
branch because it tells a story of how we build for mobile.
This work is part of https://github.com/ooni/probe/issues/1440.
Part of https://github.com/ooni/probe/issues/1440.
Basically, let us continue to update our build infrastructure so that we can release v3.10.0-beta.
Now, it's the turn of iOS.
The main changes here are two. First, tweak some bits of the POM template
to make sure it's current, including referencing the correct repo. (We were
still referencing probe-engine in there, weirdly enough).
Second, change the android build script to create the same bundle that
mavan would build. What remains to do is uploading manually.
(Because it seems my PGP key is required, I guess this development rules
out automatically generating releases at GitHub. We will see about it
later on and possibly zap the publish-android.sh script.)
Reference issue: https://github.com/ooni/probe/issues/1437.
The release 2021.04.07-180801 has been created by me using this
diff from the v3.9.0 tag. I will update the release notes as soon
as this new release becomes available from Maven Central.
The use cases for using a proxy become more clear over time. When I
originally wrote the proxy code the idea was to use the proxy to proxy
both the communication with the backend and measurements.
It become increasingly clear that we _only_ want to proxy the
communication with the backends. Therefore, there's no point in
skipping the resolver lookup step when we use a proxy.
Part of https://github.com/ooni/probe/issues/985
When this happens, the user is expressing the intention of
explicitly testing all the input they provided.
So, disable maxRuntime in these cases.
Part of https://github.com/ooni/probe/issues/1436.
When this happens, the user is expressing the intention of
explicitly testing all the input they provided.
So, disable maxRuntime in these cases.
Part of https://github.com/ooni/probe/issues/1436.
This fixes an issue where URLs provided with --input are not
accepted by the preventMistakes filter.
The filter itself needs to execute _only_ on URLs returned
by the checkIn API, rather than on URLs returned by the
InputLoader, which may instead be user provided.
Reference issue: https://github.com/ooni/probe/issues/1435
This fixes an issue where URLs provided with --input are not
accepted by the preventMistakes filter.
The filter itself needs to execute _only_ on URLs returned
by the checkIn API, rather than on URLs returned by the
InputLoader, which may instead be user provided.
Reference issue: https://github.com/ooni/probe/issues/1435
* fix(tunnel/tor): keep tunneldir clean
This diff ensures that we don't keep the log file growing and
we also remove the temporary files created by the library we
are currently using for running tor from golang.
Part of https://github.com/ooni/probe/issues/985
* fix(session.go): tell use we're using a tunnel
* urlgetter: fix tunnel test
This diff fixes the urlgetter test suite to make sure we
are correctly testing for tunnel creation.
While there, improve the way in which we create a testing
directory and add a test for that.
Part of https://github.com/ooni/probe/issues/985.
* fix comment
* fix comment
This functionality should be helpful to test that the general
interface of the tunnel package is okay from the engine package.
Part of https://github.com/ooni/probe/issues/985
* refactor(tunnel): remove nil tunnels hack
This code was originally introduced because a tunnel could be
nil in session.go. I have verified that every invocation of
tunnel.Start is careful to ensure that we have a tunnel name
and that we don't manipulate a nil tunnel.
For this reason, I'd rather remove this tricky bit of code and
further simplify the tunnel code.
Part of https://github.com/ooni/probe/issues/985
* even better docs
* feat: create tunnel inside NewSession
We want to create the tunnel when we create the session. This change
allows us to nicely ignore the problem of creating a tunnel when we
already have a proxy, as well as the problem of locking. Everything is
happening, in fact, inside of the NewSession factory.
Modify miniooni such that --tunnel is just syntactic sugar for
--proxy, at least for now. We want, in the future, to teach the
tunnel to possibly use a socks5 proxy.
Because starting a tunnel is a slow operation, we need a context in
NewSession. This causes a bunch of places to change. Not really a big
deal except we need to propagate the changes.
Make sure that the mobile code can create a new session using a
proxy for all the APIs we support.
Make sure all tests are still green and we don't loose coverage of
the various ways in which this code could be used.
This change is part of https://github.com/ooni/probe/issues/985.
* changes after merge
* fix: only keep tests that can hopefully work
While there, identify other places where we should add more
tests or fix integration tests.
Part of https://github.com/ooni/probe/issues/985
This diff breaks the circular dependency between session and
tunnel, by introducing the concept of early session.
An early session is a session that is able to fetch the psiphon
configuration file _only_ if it's embedded in the binary.
This breaks `miniooni --tunnel=psiphon` for users who have
access to the OONI backend. They are not the users we are
writing this feature for, though, so I think this is reasonable.
At the same time, this opens up the possibility of creating
a psiphon tunnel when constructing a session, which is the
approach I was following in https://github.com/ooni/probe-cli/pull/286.
This work is part of https://github.com/ooni/probe/issues/985.
Once this diff is in, I can land https://github.com/ooni/probe-cli/pull/286.
* feat(tunnel): introduce persistent tunnel state dir
This diff introduces a persistent state directory for tunnels, so that
we can bootstrap them more quickly after the first time.
Part of https://github.com/ooni/probe/issues/985
* fix: make tunnel dir optional
We have many tests where it does not make sense to explicitly
provide a tunnel dir because we're not using tunnels.
This should simplify setting up a session.
* fix(tunnel): repair tests
* final changes
* more cleanups
We're trying to remove a circular dependency between the measurement
Session and the tunnel package. To this end, continue to reduce the
dependency scope by providing TorArgs and TorBinary directly.
Part of https://github.com/ooni/probe/issues/985
Simone Basso
Federico Ceratto
Arturo Filastò