c/ooni-probe-cli
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
5b9f701317
ooni-probe-cli/.github/workflows/linux.yml

97 lines
3.8 KiB
YAML
Raw Normal View History

feat: only publish pkgs when we're in staging branches (#556) See https://github.com/ooni/probe/issues/1506#issuecomment-949664779
2021-10-22 16:13:44 +02:00
# linux builds and publishes ooniprobe for Linux
WIP: Start preparing release v3.0.7 (#147) * Use ooni/probe-engine 0.16.0 * Update all the other dependencies * Use GitHub Actions rather than Travis CI * Automatically build and test binaries on the target OS (for Windows, macOS, Linux on amd64) * Make sure we correctly measure coverage * Make sure we use `-race` when running tests * Remove unnecessary scripts * Make sure the README is up-to-date * Write small script to update binary data and add GitHub Actions checks for it * Notice that we needed to run ./updatebindata.sh and run it * Self documenting instructions regarding cross compiling * Set version number to v3.0.7-beta Part of https://github.com/ooni/probe-engine/issues/748
2020-08-22 11:20:50 +02:00
name: linux
on:
push:
refactor(.github/workflows): linux: use ./make, run less frequently (#334) While there, flush `print`s in `./make` to have more understandable logging. Also part of https://github.com/ooni/probe/issues/1466
2021-05-06 20:44:56 +02:00
branches:
- "release/**"
feat: only publish pkgs when we're in staging branches (#556) See https://github.com/ooni/probe/issues/1506#issuecomment-949664779
2021-10-22 16:13:44 +02:00
- "ooniprobe-staging"
refactor(.github/workflows): linux: use ./make, run less frequently (#334) While there, flush `print`s in `./make` to have more understandable logging. Also part of https://github.com/ooni/probe/issues/1466
2021-05-06 20:44:56 +02:00
feat(gha/linux): upload deb packages for arm64 (#337) We're still working on https://github.com/ooni/probe/issues/1466. The idea here is to teach the GH action for Linux to publish the debian package for arm64. When this is done, we can cleanup legacy build scripts and GH actions, because there is no remaining use case for them: we now build everything using the `./make` tool.
2021-05-07 09:22:46 +02:00
jobs:
feat: build and publish debian/{armhf,i386} (#344) * feat: build and publish debian/{armhf,i386} Part of https://github.com/ooni/probe/issues/807 * zap temporary build
2021-05-11 20:16:27 +02:00
build_386:
refactor(.github/workflows): linux: use ./make, run less frequently (#334) While there, flush `print`s in `./make` to have more understandable logging. Also part of https://github.com/ooni/probe/issues/1466
2021-05-06 20:44:56 +02:00
runs-on: "ubuntu-20.04"
WIP: Start preparing release v3.0.7 (#147) * Use ooni/probe-engine 0.16.0 * Update all the other dependencies * Use GitHub Actions rather than Travis CI * Automatically build and test binaries on the target OS (for Windows, macOS, Linux on amd64) * Make sure we correctly measure coverage * Make sure we use `-race` when running tests * Remove unnecessary scripts * Make sure the README is up-to-date * Write small script to update binary data and add GitHub Actions checks for it * Notice that we needed to run ./updatebindata.sh and run it * Self documenting instructions regarding cross compiling * Set version number to v3.0.7-beta Part of https://github.com/ooni/probe-engine/issues/748
2020-08-22 11:20:50 +02:00
steps:
- uses: actions/checkout@v2
[forwardport] fix(mk): do nothing if we already have psiphon config (#605) (#606) This diff forward ports ea44e99451f345474738b9010ff791759a1f1367. Original commit message: - - - This change allows for producing cloud builds using the psiphon config files. We will add those files as build secrets. Only people in the organization and collaborators with at least "write" access could trigger builds containing such secrets. Before this change, `./mk` unconditionally attempted to clone github.com/ooni/probe-private. Now, it only checks whether we need to clone _if_ files are not already there. This allows us to use GitHub actions and secrets to copy the files in there _without_ needing to clone a private repo. Cloning a private repo would require us to include as repository secret an access token with full `repo` scope, which is a very broad scope. Instead, by using secrets to include psiphon config, we are narrowing down the secrets required to make a release build. See https://github.com/ooni/probe/issues/1878 This diff WILL require forward porting to the master branch.
2021-11-19 12:40:10 +01:00
with:
fetch-depth: 0
- run: |
echo -n $PSIPHON_CONFIG_KEY > ./internal/engine/psiphon-config.key
echo $PSIPHON_CONFIG_JSON_AGE_BASE64 | base64 -d > ./internal/engine/psiphon-config.json.age
env:
PSIPHON_CONFIG_KEY: ${{ secrets.PSIPHON_CONFIG_KEY }}
PSIPHON_CONFIG_JSON_AGE_BASE64: ${{ secrets.PSIPHON_CONFIG_JSON_AGE_BASE64 }}
- run: ./mk DEBIAN_TILDE_VERSION=$GITHUB_RUN_NUMBER ./debian/386
refactor: reorganize and to end and smoke testing (#348) 1. we can merge the e2eminiooni.yml test into the miniooni.yml test so to reduce the number of tests we run; 2. ideally we would like the smoketest.sh test to evolve and also check whether we can fetch the measurements we submitted, so start moving this script into the `./E2E` folder, add a note saying we would like to do that, and direct all the tests to run this script at its new location and with its new name (`ooniprobe.sh`). With these two changes, it's fine to remove the ooniprobe2debian.yml test in ooni/e2etesting because we're moving its functionality to this repository. (We mentioned the need to do this move in a previous TODO comment at the top of such a script.) Work part of https://github.com/ooni/probe/issues/1468
2021-05-13 18:04:47 +02:00
- run: ./E2E/ooniprobe.sh ./CLI/linux/386/ooniprobe
fix(debian): make sure we can publish all archs (#350) We are mostly good to declare a stable release. We still need to deal with https://github.com/ooni/probe/issues/1484. In this PR, we fix the aforementioned issue. These are the changes: 1. we remove the vendored `debops-ci`, and we pull it directly from `ooni/sysadmin` 2. we introduce a new script, `./CLI/linux/pubdebian`, to publish packages 3. we modify `./mk` to allow for publishing debian packages built outside of CI The latter point has been quite useful in debugging what was wrong.
2021-05-19 13:54:19 +02:00
- run: ./CLI/linux/pubdebian
feat: only publish pkgs when we're in staging branches (#556) See https://github.com/ooni/probe/issues/1506#issuecomment-949664779
2021-10-22 16:13:44 +02:00
if: github.ref == 'refs/heads/ooniprobe-staging'
refactor(.github/workflows): linux: use ./make, run less frequently (#334) While there, flush `print`s in `./make` to have more understandable logging. Also part of https://github.com/ooni/probe/issues/1466
2021-05-06 20:44:56 +02:00
env:
feat: build and publish debian/{armhf,i386} (#344) * feat: build and publish debian/{armhf,i386} Part of https://github.com/ooni/probe/issues/807 * zap temporary build
2021-05-11 20:16:27 +02:00
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
DEB_GPG_KEY: ${{ secrets.DEB_GPG_KEY }}
build_amd64:
runs-on: "ubuntu-20.04"
steps:
- uses: actions/checkout@v2
[forwardport] fix(mk): do nothing if we already have psiphon config (#605) (#606) This diff forward ports ea44e99451f345474738b9010ff791759a1f1367. Original commit message: - - - This change allows for producing cloud builds using the psiphon config files. We will add those files as build secrets. Only people in the organization and collaborators with at least "write" access could trigger builds containing such secrets. Before this change, `./mk` unconditionally attempted to clone github.com/ooni/probe-private. Now, it only checks whether we need to clone _if_ files are not already there. This allows us to use GitHub actions and secrets to copy the files in there _without_ needing to clone a private repo. Cloning a private repo would require us to include as repository secret an access token with full `repo` scope, which is a very broad scope. Instead, by using secrets to include psiphon config, we are narrowing down the secrets required to make a release build. See https://github.com/ooni/probe/issues/1878 This diff WILL require forward porting to the master branch.
2021-11-19 12:40:10 +01:00
with:
fetch-depth: 0
- run: |
echo -n $PSIPHON_CONFIG_KEY > ./internal/engine/psiphon-config.key
echo $PSIPHON_CONFIG_JSON_AGE_BASE64 | base64 -d > ./internal/engine/psiphon-config.json.age
env:
PSIPHON_CONFIG_KEY: ${{ secrets.PSIPHON_CONFIG_KEY }}
PSIPHON_CONFIG_JSON_AGE_BASE64: ${{ secrets.PSIPHON_CONFIG_JSON_AGE_BASE64 }}
- run: ./mk DEBIAN_TILDE_VERSION=$GITHUB_RUN_NUMBER ./debian/amd64
refactor: reorganize and to end and smoke testing (#348) 1. we can merge the e2eminiooni.yml test into the miniooni.yml test so to reduce the number of tests we run; 2. ideally we would like the smoketest.sh test to evolve and also check whether we can fetch the measurements we submitted, so start moving this script into the `./E2E` folder, add a note saying we would like to do that, and direct all the tests to run this script at its new location and with its new name (`ooniprobe.sh`). With these two changes, it's fine to remove the ooniprobe2debian.yml test in ooni/e2etesting because we're moving its functionality to this repository. (We mentioned the need to do this move in a previous TODO comment at the top of such a script.) Work part of https://github.com/ooni/probe/issues/1468
2021-05-13 18:04:47 +02:00
- run: ./E2E/ooniprobe.sh ./CLI/linux/amd64/ooniprobe
fix(debian): make sure we can publish all archs (#350) We are mostly good to declare a stable release. We still need to deal with https://github.com/ooni/probe/issues/1484. In this PR, we fix the aforementioned issue. These are the changes: 1. we remove the vendored `debops-ci`, and we pull it directly from `ooni/sysadmin` 2. we introduce a new script, `./CLI/linux/pubdebian`, to publish packages 3. we modify `./mk` to allow for publishing debian packages built outside of CI The latter point has been quite useful in debugging what was wrong.
2021-05-19 13:54:19 +02:00
- run: ./CLI/linux/pubdebian
feat: only publish pkgs when we're in staging branches (#556) See https://github.com/ooni/probe/issues/1506#issuecomment-949664779
2021-10-22 16:13:44 +02:00
if: github.ref == 'refs/heads/ooniprobe-staging'
feat(gha/linux): upload deb packages for arm64 (#337) We're still working on https://github.com/ooni/probe/issues/1466. The idea here is to teach the GH action for Linux to publish the debian package for arm64. When this is done, we can cleanup legacy build scripts and GH actions, because there is no remaining use case for them: we now build everything using the `./make` tool.
2021-05-07 09:22:46 +02:00
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
DEB_GPG_KEY: ${{ secrets.DEB_GPG_KEY }}
refactor(.github/workflows): linux: use ./make, run less frequently (#334) While there, flush `print`s in `./make` to have more understandable logging. Also part of https://github.com/ooni/probe/issues/1466
2021-05-06 20:44:56 +02:00
feat: build and publish debian/{armhf,i386} (#344) * feat: build and publish debian/{armhf,i386} Part of https://github.com/ooni/probe/issues/807 * zap temporary build
2021-05-11 20:16:27 +02:00
build_arm:
refactor(.github/workflows): linux: use ./make, run less frequently (#334) While there, flush `print`s in `./make` to have more understandable logging. Also part of https://github.com/ooni/probe/issues/1466
2021-05-06 20:44:56 +02:00
runs-on: "ubuntu-20.04"
steps:
- uses: actions/checkout@v2
[forwardport] fix(mk): do nothing if we already have psiphon config (#605) (#606) This diff forward ports ea44e99451f345474738b9010ff791759a1f1367. Original commit message: - - - This change allows for producing cloud builds using the psiphon config files. We will add those files as build secrets. Only people in the organization and collaborators with at least "write" access could trigger builds containing such secrets. Before this change, `./mk` unconditionally attempted to clone github.com/ooni/probe-private. Now, it only checks whether we need to clone _if_ files are not already there. This allows us to use GitHub actions and secrets to copy the files in there _without_ needing to clone a private repo. Cloning a private repo would require us to include as repository secret an access token with full `repo` scope, which is a very broad scope. Instead, by using secrets to include psiphon config, we are narrowing down the secrets required to make a release build. See https://github.com/ooni/probe/issues/1878 This diff WILL require forward porting to the master branch.
2021-11-19 12:40:10 +01:00
with:
fetch-depth: 0
refactor: replace ./make (python3) with ./mk (makefile) (#343) This pull request fixes https://github.com/ooni/probe/issues/1471. We have replaced the original build script (`./make`) with the `./mk` makefile (executable using `#!/usr/bin/make -f`). We concluded supporting direct builds from Windows is not worth the effort and halving the code we need to maintain is probably a good plus. Both macOS and Linux install GNU make at `/usr/bin/make`, so we should be okay in the common use cases. I significantly simplified the management of Go versioning by requiring the user to manage it and by enforcing that we are using the desired Go version. This speeds up builds and works in sane operating systems that use the last version of a specific package. Otherwise, it's possible to use the `go get golang.org/dl/go${version}` feature. The remaining question mark was related to updating the Android SDK. I have determined that a good course of action is pinning to the latest CLI tools and always forcing the CLI tools to install the latest required packages (e.g., the NDK).
2021-05-11 16:15:13 +02:00
- run: sudo apt-get update -q
- run: sudo apt-get install -y qemu-user-static
[forwardport] fix(mk): do nothing if we already have psiphon config (#605) (#606) This diff forward ports ea44e99451f345474738b9010ff791759a1f1367. Original commit message: - - - This change allows for producing cloud builds using the psiphon config files. We will add those files as build secrets. Only people in the organization and collaborators with at least "write" access could trigger builds containing such secrets. Before this change, `./mk` unconditionally attempted to clone github.com/ooni/probe-private. Now, it only checks whether we need to clone _if_ files are not already there. This allows us to use GitHub actions and secrets to copy the files in there _without_ needing to clone a private repo. Cloning a private repo would require us to include as repository secret an access token with full `repo` scope, which is a very broad scope. Instead, by using secrets to include psiphon config, we are narrowing down the secrets required to make a release build. See https://github.com/ooni/probe/issues/1878 This diff WILL require forward porting to the master branch.
2021-11-19 12:40:10 +01:00
- run: |
echo -n $PSIPHON_CONFIG_KEY > ./internal/engine/psiphon-config.key
echo $PSIPHON_CONFIG_JSON_AGE_BASE64 | base64 -d > ./internal/engine/psiphon-config.json.age
env:
PSIPHON_CONFIG_KEY: ${{ secrets.PSIPHON_CONFIG_KEY }}
PSIPHON_CONFIG_JSON_AGE_BASE64: ${{ secrets.PSIPHON_CONFIG_JSON_AGE_BASE64 }}
- run: ./mk DEBIAN_TILDE_VERSION=$GITHUB_RUN_NUMBER ./debian/arm
refactor: reorganize and to end and smoke testing (#348) 1. we can merge the e2eminiooni.yml test into the miniooni.yml test so to reduce the number of tests we run; 2. ideally we would like the smoketest.sh test to evolve and also check whether we can fetch the measurements we submitted, so start moving this script into the `./E2E` folder, add a note saying we would like to do that, and direct all the tests to run this script at its new location and with its new name (`ooniprobe.sh`). With these two changes, it's fine to remove the ooniprobe2debian.yml test in ooni/e2etesting because we're moving its functionality to this repository. (We mentioned the need to do this move in a previous TODO comment at the top of such a script.) Work part of https://github.com/ooni/probe/issues/1468
2021-05-13 18:04:47 +02:00
- run: ./E2E/ooniprobe.sh ./CLI/linux/arm/ooniprobe
fix(debian): make sure we can publish all archs (#350) We are mostly good to declare a stable release. We still need to deal with https://github.com/ooni/probe/issues/1484. In this PR, we fix the aforementioned issue. These are the changes: 1. we remove the vendored `debops-ci`, and we pull it directly from `ooni/sysadmin` 2. we introduce a new script, `./CLI/linux/pubdebian`, to publish packages 3. we modify `./mk` to allow for publishing debian packages built outside of CI The latter point has been quite useful in debugging what was wrong.
2021-05-19 13:54:19 +02:00
- run: ./CLI/linux/pubdebian
feat: only publish pkgs when we're in staging branches (#556) See https://github.com/ooni/probe/issues/1506#issuecomment-949664779
2021-10-22 16:13:44 +02:00
if: github.ref == 'refs/heads/ooniprobe-staging'
refactor(.github/workflows): linux: use ./make, run less frequently (#334) While there, flush `print`s in `./make` to have more understandable logging. Also part of https://github.com/ooni/probe/issues/1466
2021-05-06 20:44:56 +02:00
env:
feat: build and publish debian/{armhf,i386} (#344) * feat: build and publish debian/{armhf,i386} Part of https://github.com/ooni/probe/issues/807 * zap temporary build
2021-05-11 20:16:27 +02:00
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
DEB_GPG_KEY: ${{ secrets.DEB_GPG_KEY }}
build_arm64:
runs-on: "ubuntu-20.04"
steps:
- uses: actions/checkout@v2
[forwardport] fix(mk): do nothing if we already have psiphon config (#605) (#606) This diff forward ports ea44e99451f345474738b9010ff791759a1f1367. Original commit message: - - - This change allows for producing cloud builds using the psiphon config files. We will add those files as build secrets. Only people in the organization and collaborators with at least "write" access could trigger builds containing such secrets. Before this change, `./mk` unconditionally attempted to clone github.com/ooni/probe-private. Now, it only checks whether we need to clone _if_ files are not already there. This allows us to use GitHub actions and secrets to copy the files in there _without_ needing to clone a private repo. Cloning a private repo would require us to include as repository secret an access token with full `repo` scope, which is a very broad scope. Instead, by using secrets to include psiphon config, we are narrowing down the secrets required to make a release build. See https://github.com/ooni/probe/issues/1878 This diff WILL require forward porting to the master branch.
2021-11-19 12:40:10 +01:00
with:
fetch-depth: 0
feat: build and publish debian/{armhf,i386} (#344) * feat: build and publish debian/{armhf,i386} Part of https://github.com/ooni/probe/issues/807 * zap temporary build
2021-05-11 20:16:27 +02:00
- run: sudo apt-get update -q
- run: sudo apt-get install -y qemu-user-static
[forwardport] fix(mk): do nothing if we already have psiphon config (#605) (#606) This diff forward ports ea44e99451f345474738b9010ff791759a1f1367. Original commit message: - - - This change allows for producing cloud builds using the psiphon config files. We will add those files as build secrets. Only people in the organization and collaborators with at least "write" access could trigger builds containing such secrets. Before this change, `./mk` unconditionally attempted to clone github.com/ooni/probe-private. Now, it only checks whether we need to clone _if_ files are not already there. This allows us to use GitHub actions and secrets to copy the files in there _without_ needing to clone a private repo. Cloning a private repo would require us to include as repository secret an access token with full `repo` scope, which is a very broad scope. Instead, by using secrets to include psiphon config, we are narrowing down the secrets required to make a release build. See https://github.com/ooni/probe/issues/1878 This diff WILL require forward porting to the master branch.
2021-11-19 12:40:10 +01:00
- run: |
echo -n $PSIPHON_CONFIG_KEY > ./internal/engine/psiphon-config.key
echo $PSIPHON_CONFIG_JSON_AGE_BASE64 | base64 -d > ./internal/engine/psiphon-config.json.age
env:
PSIPHON_CONFIG_KEY: ${{ secrets.PSIPHON_CONFIG_KEY }}
PSIPHON_CONFIG_JSON_AGE_BASE64: ${{ secrets.PSIPHON_CONFIG_JSON_AGE_BASE64 }}
- run: ./mk DEBIAN_TILDE_VERSION=$GITHUB_RUN_NUMBER ./debian/arm64
refactor: reorganize and to end and smoke testing (#348) 1. we can merge the e2eminiooni.yml test into the miniooni.yml test so to reduce the number of tests we run; 2. ideally we would like the smoketest.sh test to evolve and also check whether we can fetch the measurements we submitted, so start moving this script into the `./E2E` folder, add a note saying we would like to do that, and direct all the tests to run this script at its new location and with its new name (`ooniprobe.sh`). With these two changes, it's fine to remove the ooniprobe2debian.yml test in ooni/e2etesting because we're moving its functionality to this repository. (We mentioned the need to do this move in a previous TODO comment at the top of such a script.) Work part of https://github.com/ooni/probe/issues/1468
2021-05-13 18:04:47 +02:00
- run: ./E2E/ooniprobe.sh ./CLI/linux/arm64/ooniprobe
fix(debian): make sure we can publish all archs (#350) We are mostly good to declare a stable release. We still need to deal with https://github.com/ooni/probe/issues/1484. In this PR, we fix the aforementioned issue. These are the changes: 1. we remove the vendored `debops-ci`, and we pull it directly from `ooni/sysadmin` 2. we introduce a new script, `./CLI/linux/pubdebian`, to publish packages 3. we modify `./mk` to allow for publishing debian packages built outside of CI The latter point has been quite useful in debugging what was wrong.
2021-05-19 13:54:19 +02:00
- run: ./CLI/linux/pubdebian
feat: only publish pkgs when we're in staging branches (#556) See https://github.com/ooni/probe/issues/1506#issuecomment-949664779
2021-10-22 16:13:44 +02:00
if: github.ref == 'refs/heads/ooniprobe-staging'
feat(gha/linux): upload deb packages for arm64 (#337) We're still working on https://github.com/ooni/probe/issues/1466. The idea here is to teach the GH action for Linux to publish the debian package for arm64. When this is done, we can cleanup legacy build scripts and GH actions, because there is no remaining use case for them: we now build everything using the `./make` tool.
2021-05-07 09:22:46 +02:00
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
DEB_GPG_KEY: ${{ secrets.DEB_GPG_KEY }}
Reference in New Issue Copy Permalink