c/ooni-probe-cli
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
fce6eb779b
ooni-probe-cli/.github/workflows/qa.yml

17 lines
282 B
YAML
Raw Normal View History

QA: refactor, cleanup, fix, and simplify (#938) While working on https://github.com/ooni/probe/issues/2130, and specifically on the action item related to making sure all workflows are green, I was confronted with the complexity of the QA directory. There's plenty of cleaning up and simplifying there. The original intent was to A/B test `miniooni` and `measurement_kit` to ensure they were behaving the same. We don't have this need anymore. Rather, it seems the QA scripts have grown large and flaky, to the point that I am always tempted to ignore them. The underlying censorship engine, jafar, has also not been developed for quite some time. So, the first step towards improve the QA infrastructure seems to be humble and acknowledge that we cannot realistically maintain these checks using jafar as a backend for so many experiments. Let us focus on our most important experiment, Web Connectivity, and let us keep QA checks for it. Additionally, let us simplify and cleanup QA as much as possible, though without introducing radical changes. The end result is a QA for Web Connectivity that seems reasonable and runs in six minutes.
2022-09-07 14:55:36 +02:00
# Runs quality assurance checks
name: "qa"
chore: merge probe-engine into probe-cli (#201) This is how I did it: 1. `git clone https://github.com/ooni/probe-engine internal/engine` 2. ``` (cd internal/engine && git describe --tags) v0.23.0 ``` 3. `nvim go.mod` (merging `go.mod` with `internal/engine/go.mod` 4. `rm -rf internal/.git internal/engine/go.{mod,sum}` 5. `git add internal/engine` 6. `find . -type f -name \*.go -exec sed -i 's@/ooni/probe-engine@/ooni/probe-cli/v3/internal/engine@g' {} \;` 7. `go build ./...` (passes) 8. `go test -race ./...` (temporary failure on RiseupVPN) 9. `go mod tidy` 10. this commit message Once this piece of work is done, we can build a new version of `ooniprobe` that is using `internal/engine` directly. We need to do more work to ensure all the other functionality in `probe-engine` (e.g. making mobile packages) are still WAI. Part of https://github.com/ooni/probe/issues/1335
2021-02-02 12:05:47 +01:00
on:
push:
branches:
refactor(.github/workflows): reorganize many builds (#332) 1. reduce the number of periodic builds 2. just build as part of the release process in most cases 3. shorttests duplicates coverage Preliminary changes as part of https://github.com/ooni/probe/issues/1466
2021-05-06 19:56:55 +02:00
- "release/**"
feat: build ooniprobe for armv6 (#904) Part of https://github.com/ooni/probe/issues/1753. While there, introduce a rule by which, if the branch is named `fullbuild` we run all possible builds. It helps to test all the builds without creating a release branch. Because release branches are protected, they cannot be deleted easily. On the contrary, the `fullbuild` branch can easily be disposed of.
2022-08-29 17:33:42 +02:00
- "fullbuild"
QA: refactor, cleanup, fix, and simplify (#938) While working on https://github.com/ooni/probe/issues/2130, and specifically on the action item related to making sure all workflows are green, I was confronted with the complexity of the QA directory. There's plenty of cleaning up and simplifying there. The original intent was to A/B test `miniooni` and `measurement_kit` to ensure they were behaving the same. We don't have this need anymore. Rather, it seems the QA scripts have grown large and flaky, to the point that I am always tempted to ignore them. The underlying censorship engine, jafar, has also not been developed for quite some time. So, the first step towards improve the QA infrastructure seems to be humble and acknowledge that we cannot realistically maintain these checks using jafar as a backend for so many experiments. Let us focus on our most important experiment, Web Connectivity, and let us keep QA checks for it. Additionally, let us simplify and cleanup QA as much as possible, though without introducing radical changes. The end result is a QA for Web Connectivity that seems reasonable and runs in six minutes.
2022-09-07 14:55:36 +02:00
- "qabuild"
fix(workflows): elevate GITHUB_TOKEN permissions when needed (#822) I am not 100% sure I was able to fix all the cases in which we need higher permissions than the strict default. At least, I tried. It may be reasonable to make an interim release to check whether I successfully fixed all the cases. Ref issue: https://github.com/ooni/probe/issues/2154
2022-07-01 12:11:32 +02:00
chore: merge probe-engine into probe-cli (#201) This is how I did it: 1. `git clone https://github.com/ooni/probe-engine internal/engine` 2. ``` (cd internal/engine && git describe --tags) v0.23.0 ``` 3. `nvim go.mod` (merging `go.mod` with `internal/engine/go.mod` 4. `rm -rf internal/.git internal/engine/go.{mod,sum}` 5. `git add internal/engine` 6. `find . -type f -name \*.go -exec sed -i 's@/ooni/probe-engine@/ooni/probe-cli/v3/internal/engine@g' {} \;` 7. `go build ./...` (passes) 8. `go test -race ./...` (temporary failure on RiseupVPN) 9. `go mod tidy` 10. this commit message Once this piece of work is done, we can build a new version of `ooniprobe` that is using `internal/engine` directly. We need to do more work to ensure all the other functionality in `probe-engine` (e.g. making mobile packages) are still WAI. Part of https://github.com/ooni/probe/issues/1335
2021-02-02 12:05:47 +01:00
jobs:
QA: refactor, cleanup, fix, and simplify (#938) While working on https://github.com/ooni/probe/issues/2130, and specifically on the action item related to making sure all workflows are green, I was confronted with the complexity of the QA directory. There's plenty of cleaning up and simplifying there. The original intent was to A/B test `miniooni` and `measurement_kit` to ensure they were behaving the same. We don't have this need anymore. Rather, it seems the QA scripts have grown large and flaky, to the point that I am always tempted to ignore them. The underlying censorship engine, jafar, has also not been developed for quite some time. So, the first step towards improve the QA infrastructure seems to be humble and acknowledge that we cannot realistically maintain these checks using jafar as a backend for so many experiments. Let us focus on our most important experiment, Web Connectivity, and let us keep QA checks for it. Additionally, let us simplify and cleanup QA as much as possible, though without introducing radical changes. The end result is a QA for Web Connectivity that seems reasonable and runs in six minutes.
2022-09-07 14:55:36 +02:00
test_webconnectivity:
refactor(.github/workflows): reorganize many builds (#332) 1. reduce the number of periodic builds 2. just build as part of the release process in most cases 3. shorttests duplicates coverage Preliminary changes as part of https://github.com/ooni/probe/issues/1466
2021-05-06 19:56:55 +02:00
runs-on: ubuntu-20.04
chore: merge probe-engine into probe-cli (#201) This is how I did it: 1. `git clone https://github.com/ooni/probe-engine internal/engine` 2. ``` (cd internal/engine && git describe --tags) v0.23.0 ``` 3. `nvim go.mod` (merging `go.mod` with `internal/engine/go.mod` 4. `rm -rf internal/.git internal/engine/go.{mod,sum}` 5. `git add internal/engine` 6. `find . -type f -name \*.go -exec sed -i 's@/ooni/probe-engine@/ooni/probe-cli/v3/internal/engine@g' {} \;` 7. `go build ./...` (passes) 8. `go test -race ./...` (temporary failure on RiseupVPN) 9. `go mod tidy` 10. this commit message Once this piece of work is done, we can build a new version of `ooniprobe` that is using `internal/engine` directly. We need to do more work to ensure all the other functionality in `probe-engine` (e.g. making mobile packages) are still WAI. Part of https://github.com/ooni/probe/issues/1335
2021-02-02 12:05:47 +01:00
steps:
cleanup: define required Go version just once (#861) See https://github.com/ooni/probe/issues/2217
2022-08-17 11:39:38 +02:00
- uses: actions/checkout@v2
chore: merge probe-engine into probe-cli (#201) This is how I did it: 1. `git clone https://github.com/ooni/probe-engine internal/engine` 2. ``` (cd internal/engine && git describe --tags) v0.23.0 ``` 3. `nvim go.mod` (merging `go.mod` with `internal/engine/go.mod` 4. `rm -rf internal/.git internal/engine/go.{mod,sum}` 5. `git add internal/engine` 6. `find . -type f -name \*.go -exec sed -i 's@/ooni/probe-engine@/ooni/probe-cli/v3/internal/engine@g' {} \;` 7. `go build ./...` (passes) 8. `go test -race ./...` (temporary failure on RiseupVPN) 9. `go mod tidy` 10. this commit message Once this piece of work is done, we can build a new version of `ooniprobe` that is using `internal/engine` directly. We need to do more work to ensure all the other functionality in `probe-engine` (e.g. making mobile packages) are still WAI. Part of https://github.com/ooni/probe/issues/1335
2021-02-02 12:05:47 +01:00
- run: ./QA/rundocker.bash "webconnectivity"
QA: refactor, cleanup, fix, and simplify (#938) While working on https://github.com/ooni/probe/issues/2130, and specifically on the action item related to making sure all workflows are green, I was confronted with the complexity of the QA directory. There's plenty of cleaning up and simplifying there. The original intent was to A/B test `miniooni` and `measurement_kit` to ensure they were behaving the same. We don't have this need anymore. Rather, it seems the QA scripts have grown large and flaky, to the point that I am always tempted to ignore them. The underlying censorship engine, jafar, has also not been developed for quite some time. So, the first step towards improve the QA infrastructure seems to be humble and acknowledge that we cannot realistically maintain these checks using jafar as a backend for so many experiments. Let us focus on our most important experiment, Web Connectivity, and let us keep QA checks for it. Additionally, let us simplify and cleanup QA as much as possible, though without introducing radical changes. The end result is a QA for Web Connectivity that seems reasonable and runs in six minutes.
2022-09-07 14:55:36 +02:00
Reference in New Issue Copy Permalink