c/ooni-probe-cli
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

fix(measurex): use same keys of the OONI data format (#572)

Browse Source 
This change should simplify the pipeline's job.

Reference issue: https://github.com/ooni/probe/issues/1817.

I previously dismissed this possibility, but now it seems clear it
is simpler to have a very tabular data format internally and to
convert such a format to OONI's data format when serializing.

The OONI data format is what the pipeline expects, but processing
is easier with a more linear/tabular format.
This commit is contained in:
Simone Basso
2021-11-05 10:46:45 +01:00
committed by GitHub
parent 6f90d29bfa
commit aa27bbe33f
35 changed files with 1571 additions and 1025 deletions
Download Patch File Download Diff File Expand all files Collapse all files
internal/tutorial/measurex/chapter12/README.md
+5 -2
View File
@@ -32,7 +32,7 @@ import (
)
type measurement struct {
URLs []*measurex.URLMeasurement
URLs []*measurex.ArchivalURLMeasurement
}
func print(v interface{}) {
@@ -67,7 +67,7 @@ is closed when done by `MeasureURLAndFollowRedirections`, so we leave the loop.
```Go
for m := range mx.MeasureURLAndFollowRedirections(ctx, *URL, headers, cookies) {
all.URLs = append(all.URLs, m)
all.URLs = append(all.URLs, measurex.NewArchivalURLMeasurement(m))
}
print(all)
}
@@ -86,6 +86,9 @@ Take a look at the JSON. You should see several redirects
and that we measure each endpoint of each redirect, including
QUIC endpoints that we discover on the way.
Exercise: remove code for converting to OONI data format
and compare output with previous chapter. See any difference?
## Conclusion
We have introduced `MeasureURLAndFollowRedirect`, the
internal/tutorial/measurex/chapter12/main.go
+5 -2
View File
@@ -33,7 +33,7 @@ import (
)
type measurement struct {
URLs []*measurex.URLMeasurement
URLs []*measurex.ArchivalURLMeasurement
}
func print(v interface{}) {
@@ -68,7 +68,7 @@ func main() {
//
// ```Go
for m := range mx.MeasureURLAndFollowRedirections(ctx, *URL, headers, cookies) {
all.URLs = append(all.URLs, m)
all.URLs = append(all.URLs, measurex.NewArchivalURLMeasurement(m))
}
print(all)
}
@@ -87,6 +87,9 @@ func main() {
// and that we measure each endpoint of each redirect, including
// QUIC endpoints that we discover on the way.
//
// Exercise: remove code for converting to OONI data format
// and compare output with previous chapter. See any difference?
//
// ## Conclusion
//
// We have introduced `MeasureURLAndFollowRedirect`, the