This pull request fixes https://github.com/ooni/probe/issues/1471. We have replaced the original build script (`./make`) with the `./mk` makefile (executable using `#!/usr/bin/make -f`). We concluded supporting direct builds from Windows is not worth the effort and halving the code we need to maintain is probably a good plus. Both macOS and Linux install GNU make at `/usr/bin/make`, so we should be okay in the common use cases.
I significantly simplified the management of Go versioning by requiring the user to manage it and by enforcing that we are using the desired Go version. This speeds up builds and works in sane operating systems that use the last version of a specific package. Otherwise, it's possible to use the `go get golang.org/dl/go${version}` feature.
The remaining question mark was related to updating the Android SDK. I have determined that a good course of action is pinning to the latest CLI tools and always forcing the CLI tools to install the latest required packages (e.g., the NDK).
We're still working on https://github.com/ooni/probe/issues/1466. The idea here is to teach the GH action for Linux to publish the debian package for arm64. When this is done, we can cleanup legacy build scripts and GH actions, because there is no remaining use case for them: we now build everything using the `./make` tool.
Part of https://github.com/ooni/probe/issues/1466. We're building both `arm64` and `amd64`. We are still not publishing `arm64` packages, which is what is asked in the original issue, but we're really close to doing that.
This PR groups misc cleanup and changes from https://github.com/ooni/probe-cli/pull/331.
* CLI/linux/build: add documentation
* debian/.gitignore: ignore generated files
* debian/TODO: unnecessary at this point
* debian/ooniprobe-cli.service: remove commented out lines
* debian/rules: remove unnecessary actions
* make: reindent and fix spelling
* smoketest.sh: don't run in verbose mode
Part of https://github.com/ooni/probe/issues/1466
1. reduce the number of periodic builds
2. just build as part of the release process in most cases
3. shorttests duplicates coverage
Preliminary changes as part of https://github.com/ooni/probe/issues/1466
Just consistency. There was no clear semantic regarding why
sometimes I was using __name and sometimes _name.
So prefer _name everywhere.
Occurred to me while working on https://github.com/ooni/probe/issues/1466.
After all the refactoring done so far, we can run checks directly
inside of `make`, because we have auto-cleanup, temporary environments
and we don't need wrapper scripts anymore.
Part of https://github.com/ooni/probe/issues/1466.
The previous fix allowed us to return values directly from
implementations of Engine. Here we take advantage of this
possibility by immediately refactoring how backticks work.
Part of https://github.com/ooni/probe/issues/1466.
The previous composition strategy was such that it was difficult
to compose functions returning a value.
This new composition strategy is better because it allows us to
return values, which is definitely going to help us.
Part of https://github.com/ooni/probe/issues/1466.
* fix(tunnel): pass /absolute/path/to/tor to cretz/bine
It seems cretz/bine is not aware of https://blog.golang.org/path-security
for now. I am planning to send over a diff for that later today.
In the meanwhile, do the right thing here, and make sure that we obtain
the absolute path to the tor binary before we continue.
This work is part of https://github.com/ooni/probe-engine/issues/283.
* fix tests when tor is not installed
* Add support for tracking the is_uploaded status in the results table
* Add unit tests for measurement upload status
This implements: https://github.com/ooni/probe/issues/1457
* Update cmd/ooniprobe/internal/database/actions.go
Co-authored-by: Simone Basso <bassosimone@gmail.com>
* fix(make): correctly write oonimkall.podspec
Part of https://github.com/ooni/probe/issues/1439
* chore: set version number to v3.10.0-beta.1
* fix(ios): don't build a target that requires git
This commit merges the mobile-staging branch into the main history so that
we are not losing track of what it happened during its lifetime.
We're going to not need this branch anymore, because we're now going to
build releases on a special purpose build machine.
At the same time, we don't want to loose track of the history of such a
branch because it tells a story of how we build for mobile.
This work is part of https://github.com/ooni/probe/issues/1440.
Part of https://github.com/ooni/probe/issues/1440.
Basically, let us continue to update our build infrastructure so that we can release v3.10.0-beta.
Now, it's the turn of iOS.
The main changes here are two. First, tweak some bits of the POM template
to make sure it's current, including referencing the correct repo. (We were
still referencing probe-engine in there, weirdly enough).
Second, change the android build script to create the same bundle that
mavan would build. What remains to do is uploading manually.
(Because it seems my PGP key is required, I guess this development rules
out automatically generating releases at GitHub. We will see about it
later on and possibly zap the publish-android.sh script.)
Reference issue: https://github.com/ooni/probe/issues/1437.
The release 2021.04.07-180801 has been created by me using this
diff from the v3.9.0 tag. I will update the release notes as soon
as this new release becomes available from Maven Central.
The use cases for using a proxy become more clear over time. When I
originally wrote the proxy code the idea was to use the proxy to proxy
both the communication with the backend and measurements.
It become increasingly clear that we _only_ want to proxy the
communication with the backends. Therefore, there's no point in
skipping the resolver lookup step when we use a proxy.
Part of https://github.com/ooni/probe/issues/985
When this happens, the user is expressing the intention of
explicitly testing all the input they provided.
So, disable maxRuntime in these cases.
Part of https://github.com/ooni/probe/issues/1436.
When this happens, the user is expressing the intention of
explicitly testing all the input they provided.
So, disable maxRuntime in these cases.
Part of https://github.com/ooni/probe/issues/1436.
This fixes an issue where URLs provided with --input are not
accepted by the preventMistakes filter.
The filter itself needs to execute _only_ on URLs returned
by the checkIn API, rather than on URLs returned by the
InputLoader, which may instead be user provided.
Reference issue: https://github.com/ooni/probe/issues/1435
This fixes an issue where URLs provided with --input are not
accepted by the preventMistakes filter.
The filter itself needs to execute _only_ on URLs returned
by the checkIn API, rather than on URLs returned by the
InputLoader, which may instead be user provided.
Reference issue: https://github.com/ooni/probe/issues/1435
* fix(tunnel/tor): keep tunneldir clean
This diff ensures that we don't keep the log file growing and
we also remove the temporary files created by the library we
are currently using for running tor from golang.
Part of https://github.com/ooni/probe/issues/985
* fix(session.go): tell use we're using a tunnel
* urlgetter: fix tunnel test
This diff fixes the urlgetter test suite to make sure we
are correctly testing for tunnel creation.
While there, improve the way in which we create a testing
directory and add a test for that.
Part of https://github.com/ooni/probe/issues/985.
* fix comment
* fix comment
Simone Basso
Federico Ceratto
Arturo Filastò