c/ooni-probe-cli
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

fix(measurex): allow API user to choose parallelism (#581)

Browse Source 
Closes https://github.com/ooni/probe/issues/1818
This commit is contained in:
Simone Basso
2021-11-05 14:37:03 +01:00
committed by GitHub
parent 3b27780836
commit ba7b981fcb
11 changed files with 88 additions and 34 deletions
Download Patch File Download Diff File Expand all files Collapse all files
internal/tutorial/measurex/chapter12/README.md
+6 -1
View File
@@ -62,11 +62,16 @@ returns a channel where it posts the result of measuring
the original URL along with all its redirections. Internally,
`MeasureURLAndFollowRedirections` calls `MeasureURL`.
The parallelism argument dictates how many parallel goroutine
to use for parallelizable operations. (A zero or negative
value implies that the code should use a sensible default value.)
We accumulate the results in `URLs` and print `m`. The channel
is closed when done by `MeasureURLAndFollowRedirections`, so we leave the loop.
```Go
for m := range mx.MeasureURLAndFollowRedirections(ctx, *URL, headers, cookies) {
const parallelism = 3
for m := range mx.MeasureURLAndFollowRedirections(ctx, parallelism, *URL, headers, cookies) {
all.URLs = append(all.URLs, measurex.NewArchivalURLMeasurement(m))
}
print(all)
internal/tutorial/measurex/chapter12/main.go
+6 -1
View File
@@ -63,11 +63,16 @@ func main() {
// the original URL along with all its redirections. Internally,
// `MeasureURLAndFollowRedirections` calls `MeasureURL`.
//
// The parallelism argument dictates how many parallel goroutine
// to use for parallelizable operations. (A zero or negative
// value implies that the code should use a sensible default value.)
//
// We accumulate the results in `URLs` and print `m`. The channel
// is closed when done by `MeasureURLAndFollowRedirections`, so we leave the loop.
//
// ```Go
for m := range mx.MeasureURLAndFollowRedirections(ctx, *URL, headers, cookies) {
const parallelism = 3
for m := range mx.MeasureURLAndFollowRedirections(ctx, parallelism, *URL, headers, cookies) {
all.URLs = append(all.URLs, measurex.NewArchivalURLMeasurement(m))
}
print(all)