feat: introduce ptx package for pluggable transports dialers (#373)
* feat: introduce ptx package for pluggable transports dialers
Version 2 of the pluggable transports specification defines a function
that's like `Dial() (net.Conn, error`).
Because we use contexts as much as possible in `probe-cli`, we are
wrapping such an interface into a `DialContext` func.
The code for obfs4 is adapted from https://github.com/ooni/probe-cli/pull/341.
The code for snowflake is significantly easier than it is in
https://github.com/ooni/probe-cli/pull/341, because now Snowflake
supports the PTv2 spec (thanks @cohosh!).
The code for setting up a pluggable transport listener has also
been adapted from https://github.com/ooni/probe-cli/pull/341.
We cannot merge this code yet, because we need unit testing, yet the
newly added code already seems suitable for these use cases:
1. testing by dialing and seeing whether we can dial (which is not
very useful but still better than not doing it);
2. spawning tor+pluggable transports for circumvention (we need a
little more hammering like we did in https://github.com/ooni/probe-cli/pull/341,
which is basically https://github.com/ooni/probe/issues/1565, and then
we will be able to do that, as demonstrated by the new, simple client which
already allows us to use pluggable transports with tor);
3. testing by launching tor (when available) with a set of
pluggable transports (which depends on https://github.com/ooni/probe-engine/issues/897
and has not been assigned an issue yet).
* fix: tweaks after self code-review
* feat: write quick tests for ptx/obfs4
(They run in 0.4s, so I think it's fine for them to always run.)
* feat(ptx/snowflake): write unit and integration tests
* feat: create a fake PTDialer
The idea is that we'll use this simpler PTDialer for testing.
* feat: finish writing tests for new package
* Apply suggestions from code review
* Update internal/ptx/dependencies_test.go
Co-authored-by: Arturo Filastò <arturo@openobservatory.org>
* Update internal/ptx/dependencies_test.go
Co-authored-by: Arturo Filastò <arturo@openobservatory.org>
* chore: use as testing bridge one that's used by tor browser
The previous testing bridge used to be used by tor browser but
it was subsequently removed here:
https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/commit/e26e91bef8bd8d04d79bdd69f087efd808bc925d
See https://github.com/ooni/probe-cli/pull/373#discussion_r649820724
Co-authored-by: Arturo Filastò <arturo@openobservatory.org>
2021-06-14 10:20:54 +02:00
|
|
|
package ptx
|
|
|
|
|
|
|
|
import (
|
|
|
|
"context"
|
|
|
|
"errors"
|
|
|
|
"net"
|
|
|
|
"strings"
|
|
|
|
"testing"
|
|
|
|
|
|
|
|
"github.com/ooni/probe-cli/v3/internal/atomicx"
|
2021-06-23 16:06:02 +02:00
|
|
|
"github.com/ooni/probe-cli/v3/internal/netxmocks"
|
feat: introduce ptx package for pluggable transports dialers (#373)
* feat: introduce ptx package for pluggable transports dialers
Version 2 of the pluggable transports specification defines a function
that's like `Dial() (net.Conn, error`).
Because we use contexts as much as possible in `probe-cli`, we are
wrapping such an interface into a `DialContext` func.
The code for obfs4 is adapted from https://github.com/ooni/probe-cli/pull/341.
The code for snowflake is significantly easier than it is in
https://github.com/ooni/probe-cli/pull/341, because now Snowflake
supports the PTv2 spec (thanks @cohosh!).
The code for setting up a pluggable transport listener has also
been adapted from https://github.com/ooni/probe-cli/pull/341.
We cannot merge this code yet, because we need unit testing, yet the
newly added code already seems suitable for these use cases:
1. testing by dialing and seeing whether we can dial (which is not
very useful but still better than not doing it);
2. spawning tor+pluggable transports for circumvention (we need a
little more hammering like we did in https://github.com/ooni/probe-cli/pull/341,
which is basically https://github.com/ooni/probe/issues/1565, and then
we will be able to do that, as demonstrated by the new, simple client which
already allows us to use pluggable transports with tor);
3. testing by launching tor (when available) with a set of
pluggable transports (which depends on https://github.com/ooni/probe-engine/issues/897
and has not been assigned an issue yet).
* fix: tweaks after self code-review
* feat: write quick tests for ptx/obfs4
(They run in 0.4s, so I think it's fine for them to always run.)
* feat(ptx/snowflake): write unit and integration tests
* feat: create a fake PTDialer
The idea is that we'll use this simpler PTDialer for testing.
* feat: finish writing tests for new package
* Apply suggestions from code review
* Update internal/ptx/dependencies_test.go
Co-authored-by: Arturo Filastò <arturo@openobservatory.org>
* Update internal/ptx/dependencies_test.go
Co-authored-by: Arturo Filastò <arturo@openobservatory.org>
* chore: use as testing bridge one that's used by tor browser
The previous testing bridge used to be used by tor browser but
it was subsequently removed here:
https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/commit/e26e91bef8bd8d04d79bdd69f087efd808bc925d
See https://github.com/ooni/probe-cli/pull/373#discussion_r649820724
Co-authored-by: Arturo Filastò <arturo@openobservatory.org>
2021-06-14 10:20:54 +02:00
|
|
|
)
|
|
|
|
|
|
|
|
func TestOBFS4DialerWorks(t *testing.T) {
|
|
|
|
// This test is 0.3 seconds in my machine, so it's ~fine
|
|
|
|
// to run it even when we're in short mode
|
|
|
|
o4d := DefaultTestingOBFS4Bridge()
|
|
|
|
conn, err := o4d.DialContext(context.Background())
|
|
|
|
if err != nil {
|
|
|
|
t.Fatal(err)
|
|
|
|
}
|
|
|
|
if conn == nil {
|
|
|
|
t.Fatal("expected non-nil conn here")
|
|
|
|
}
|
|
|
|
if o4d.Name() != "obfs4" {
|
|
|
|
t.Fatal("unexpected value returned by Name")
|
|
|
|
}
|
|
|
|
bridgearg := o4d.AsBridgeArgument()
|
2021-07-01 15:13:01 +02:00
|
|
|
expectedbridge := "obfs4 209.148.46.65:443 74FAD13168806246602538555B5521A0383A1875 cert=ssH+9rP8dG2NLDN2XuFw63hIO/9MNNinLmxQDpVa+7kTOa9/m+tGWT1SmSYpQ9uTBGa6Hw iat-mode=0"
|
feat: introduce ptx package for pluggable transports dialers (#373)
* feat: introduce ptx package for pluggable transports dialers
Version 2 of the pluggable transports specification defines a function
that's like `Dial() (net.Conn, error`).
Because we use contexts as much as possible in `probe-cli`, we are
wrapping such an interface into a `DialContext` func.
The code for obfs4 is adapted from https://github.com/ooni/probe-cli/pull/341.
The code for snowflake is significantly easier than it is in
https://github.com/ooni/probe-cli/pull/341, because now Snowflake
supports the PTv2 spec (thanks @cohosh!).
The code for setting up a pluggable transport listener has also
been adapted from https://github.com/ooni/probe-cli/pull/341.
We cannot merge this code yet, because we need unit testing, yet the
newly added code already seems suitable for these use cases:
1. testing by dialing and seeing whether we can dial (which is not
very useful but still better than not doing it);
2. spawning tor+pluggable transports for circumvention (we need a
little more hammering like we did in https://github.com/ooni/probe-cli/pull/341,
which is basically https://github.com/ooni/probe/issues/1565, and then
we will be able to do that, as demonstrated by the new, simple client which
already allows us to use pluggable transports with tor);
3. testing by launching tor (when available) with a set of
pluggable transports (which depends on https://github.com/ooni/probe-engine/issues/897
and has not been assigned an issue yet).
* fix: tweaks after self code-review
* feat: write quick tests for ptx/obfs4
(They run in 0.4s, so I think it's fine for them to always run.)
* feat(ptx/snowflake): write unit and integration tests
* feat: create a fake PTDialer
The idea is that we'll use this simpler PTDialer for testing.
* feat: finish writing tests for new package
* Apply suggestions from code review
* Update internal/ptx/dependencies_test.go
Co-authored-by: Arturo Filastò <arturo@openobservatory.org>
* Update internal/ptx/dependencies_test.go
Co-authored-by: Arturo Filastò <arturo@openobservatory.org>
* chore: use as testing bridge one that's used by tor browser
The previous testing bridge used to be used by tor browser but
it was subsequently removed here:
https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/commit/e26e91bef8bd8d04d79bdd69f087efd808bc925d
See https://github.com/ooni/probe-cli/pull/373#discussion_r649820724
Co-authored-by: Arturo Filastò <arturo@openobservatory.org>
2021-06-14 10:20:54 +02:00
|
|
|
if bridgearg != expectedbridge {
|
|
|
|
t.Fatal("unexpected AsBridgeArgument value", bridgearg)
|
|
|
|
}
|
|
|
|
conn.Close()
|
|
|
|
}
|
|
|
|
|
|
|
|
func TestOBFS4DialerFailsWithInvalidCert(t *testing.T) {
|
|
|
|
o4d := DefaultTestingOBFS4Bridge()
|
|
|
|
o4d.Cert = "antani!!!"
|
|
|
|
conn, err := o4d.DialContext(context.Background())
|
|
|
|
if err == nil || !strings.HasPrefix(err.Error(), "failed to decode cert:") {
|
|
|
|
t.Fatal("not the error we expected", err)
|
|
|
|
}
|
|
|
|
if conn != nil {
|
|
|
|
t.Fatal("expected nil conn here")
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
func TestOBFS4DialerFailsWithConnectionErrorAndNoContextExpiration(t *testing.T) {
|
|
|
|
expected := errors.New("mocked error")
|
|
|
|
o4d := DefaultTestingOBFS4Bridge()
|
2021-06-23 16:06:02 +02:00
|
|
|
o4d.UnderlyingDialer = &netxmocks.Dialer{
|
feat: introduce ptx package for pluggable transports dialers (#373)
* feat: introduce ptx package for pluggable transports dialers
Version 2 of the pluggable transports specification defines a function
that's like `Dial() (net.Conn, error`).
Because we use contexts as much as possible in `probe-cli`, we are
wrapping such an interface into a `DialContext` func.
The code for obfs4 is adapted from https://github.com/ooni/probe-cli/pull/341.
The code for snowflake is significantly easier than it is in
https://github.com/ooni/probe-cli/pull/341, because now Snowflake
supports the PTv2 spec (thanks @cohosh!).
The code for setting up a pluggable transport listener has also
been adapted from https://github.com/ooni/probe-cli/pull/341.
We cannot merge this code yet, because we need unit testing, yet the
newly added code already seems suitable for these use cases:
1. testing by dialing and seeing whether we can dial (which is not
very useful but still better than not doing it);
2. spawning tor+pluggable transports for circumvention (we need a
little more hammering like we did in https://github.com/ooni/probe-cli/pull/341,
which is basically https://github.com/ooni/probe/issues/1565, and then
we will be able to do that, as demonstrated by the new, simple client which
already allows us to use pluggable transports with tor);
3. testing by launching tor (when available) with a set of
pluggable transports (which depends on https://github.com/ooni/probe-engine/issues/897
and has not been assigned an issue yet).
* fix: tweaks after self code-review
* feat: write quick tests for ptx/obfs4
(They run in 0.4s, so I think it's fine for them to always run.)
* feat(ptx/snowflake): write unit and integration tests
* feat: create a fake PTDialer
The idea is that we'll use this simpler PTDialer for testing.
* feat: finish writing tests for new package
* Apply suggestions from code review
* Update internal/ptx/dependencies_test.go
Co-authored-by: Arturo Filastò <arturo@openobservatory.org>
* Update internal/ptx/dependencies_test.go
Co-authored-by: Arturo Filastò <arturo@openobservatory.org>
* chore: use as testing bridge one that's used by tor browser
The previous testing bridge used to be used by tor browser but
it was subsequently removed here:
https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/commit/e26e91bef8bd8d04d79bdd69f087efd808bc925d
See https://github.com/ooni/probe-cli/pull/373#discussion_r649820724
Co-authored-by: Arturo Filastò <arturo@openobservatory.org>
2021-06-14 10:20:54 +02:00
|
|
|
MockDialContext: func(ctx context.Context, network string, address string) (net.Conn, error) {
|
|
|
|
return nil, expected
|
|
|
|
},
|
|
|
|
}
|
|
|
|
conn, err := o4d.DialContext(context.Background())
|
|
|
|
if !errors.Is(err, expected) {
|
|
|
|
t.Fatal("not the error we expected", err)
|
|
|
|
}
|
|
|
|
if conn != nil {
|
|
|
|
t.Fatal("expected nil conn here")
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
func TestOBFS4DialerFailsWithConnectionErrorAndContextExpiration(t *testing.T) {
|
|
|
|
ctx, cancel := context.WithCancel(context.Background())
|
|
|
|
defer cancel()
|
|
|
|
expected := errors.New("mocked error")
|
|
|
|
o4d := DefaultTestingOBFS4Bridge()
|
2021-06-23 16:06:02 +02:00
|
|
|
o4d.UnderlyingDialer = &netxmocks.Dialer{
|
feat: introduce ptx package for pluggable transports dialers (#373)
* feat: introduce ptx package for pluggable transports dialers
Version 2 of the pluggable transports specification defines a function
that's like `Dial() (net.Conn, error`).
Because we use contexts as much as possible in `probe-cli`, we are
wrapping such an interface into a `DialContext` func.
The code for obfs4 is adapted from https://github.com/ooni/probe-cli/pull/341.
The code for snowflake is significantly easier than it is in
https://github.com/ooni/probe-cli/pull/341, because now Snowflake
supports the PTv2 spec (thanks @cohosh!).
The code for setting up a pluggable transport listener has also
been adapted from https://github.com/ooni/probe-cli/pull/341.
We cannot merge this code yet, because we need unit testing, yet the
newly added code already seems suitable for these use cases:
1. testing by dialing and seeing whether we can dial (which is not
very useful but still better than not doing it);
2. spawning tor+pluggable transports for circumvention (we need a
little more hammering like we did in https://github.com/ooni/probe-cli/pull/341,
which is basically https://github.com/ooni/probe/issues/1565, and then
we will be able to do that, as demonstrated by the new, simple client which
already allows us to use pluggable transports with tor);
3. testing by launching tor (when available) with a set of
pluggable transports (which depends on https://github.com/ooni/probe-engine/issues/897
and has not been assigned an issue yet).
* fix: tweaks after self code-review
* feat: write quick tests for ptx/obfs4
(They run in 0.4s, so I think it's fine for them to always run.)
* feat(ptx/snowflake): write unit and integration tests
* feat: create a fake PTDialer
The idea is that we'll use this simpler PTDialer for testing.
* feat: finish writing tests for new package
* Apply suggestions from code review
* Update internal/ptx/dependencies_test.go
Co-authored-by: Arturo Filastò <arturo@openobservatory.org>
* Update internal/ptx/dependencies_test.go
Co-authored-by: Arturo Filastò <arturo@openobservatory.org>
* chore: use as testing bridge one that's used by tor browser
The previous testing bridge used to be used by tor browser but
it was subsequently removed here:
https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/commit/e26e91bef8bd8d04d79bdd69f087efd808bc925d
See https://github.com/ooni/probe-cli/pull/373#discussion_r649820724
Co-authored-by: Arturo Filastò <arturo@openobservatory.org>
2021-06-14 10:20:54 +02:00
|
|
|
MockDialContext: func(ctx context.Context, network string, address string) (net.Conn, error) {
|
|
|
|
// We cancel the context before returning the error, which makes
|
|
|
|
// the context cancellation happen before us returning.
|
|
|
|
cancel()
|
|
|
|
return nil, expected
|
|
|
|
},
|
|
|
|
}
|
|
|
|
conn, err := o4d.DialContext(ctx)
|
|
|
|
if !errors.Is(err, context.Canceled) {
|
|
|
|
t.Fatal("not the error we expected", err)
|
|
|
|
}
|
|
|
|
if conn != nil {
|
|
|
|
t.Fatal("expected nil conn here")
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
// obfs4connwrapper allows us to observe that Close has been called
|
|
|
|
type obfs4connwrapper struct {
|
|
|
|
net.Conn
|
|
|
|
called *atomicx.Int64
|
|
|
|
}
|
|
|
|
|
|
|
|
// Close implements net.Conn.Close
|
|
|
|
func (c *obfs4connwrapper) Close() error {
|
|
|
|
c.called.Add(1)
|
|
|
|
return c.Conn.Close()
|
|
|
|
}
|
|
|
|
|
|
|
|
func TestOBFS4DialerWorksWithContextExpiration(t *testing.T) {
|
|
|
|
ctx, cancel := context.WithCancel(context.Background())
|
|
|
|
defer cancel()
|
|
|
|
called := &atomicx.Int64{}
|
|
|
|
o4d := DefaultTestingOBFS4Bridge()
|
2021-06-23 16:06:02 +02:00
|
|
|
o4d.UnderlyingDialer = &netxmocks.Dialer{
|
feat: introduce ptx package for pluggable transports dialers (#373)
* feat: introduce ptx package for pluggable transports dialers
Version 2 of the pluggable transports specification defines a function
that's like `Dial() (net.Conn, error`).
Because we use contexts as much as possible in `probe-cli`, we are
wrapping such an interface into a `DialContext` func.
The code for obfs4 is adapted from https://github.com/ooni/probe-cli/pull/341.
The code for snowflake is significantly easier than it is in
https://github.com/ooni/probe-cli/pull/341, because now Snowflake
supports the PTv2 spec (thanks @cohosh!).
The code for setting up a pluggable transport listener has also
been adapted from https://github.com/ooni/probe-cli/pull/341.
We cannot merge this code yet, because we need unit testing, yet the
newly added code already seems suitable for these use cases:
1. testing by dialing and seeing whether we can dial (which is not
very useful but still better than not doing it);
2. spawning tor+pluggable transports for circumvention (we need a
little more hammering like we did in https://github.com/ooni/probe-cli/pull/341,
which is basically https://github.com/ooni/probe/issues/1565, and then
we will be able to do that, as demonstrated by the new, simple client which
already allows us to use pluggable transports with tor);
3. testing by launching tor (when available) with a set of
pluggable transports (which depends on https://github.com/ooni/probe-engine/issues/897
and has not been assigned an issue yet).
* fix: tweaks after self code-review
* feat: write quick tests for ptx/obfs4
(They run in 0.4s, so I think it's fine for them to always run.)
* feat(ptx/snowflake): write unit and integration tests
* feat: create a fake PTDialer
The idea is that we'll use this simpler PTDialer for testing.
* feat: finish writing tests for new package
* Apply suggestions from code review
* Update internal/ptx/dependencies_test.go
Co-authored-by: Arturo Filastò <arturo@openobservatory.org>
* Update internal/ptx/dependencies_test.go
Co-authored-by: Arturo Filastò <arturo@openobservatory.org>
* chore: use as testing bridge one that's used by tor browser
The previous testing bridge used to be used by tor browser but
it was subsequently removed here:
https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/commit/e26e91bef8bd8d04d79bdd69f087efd808bc925d
See https://github.com/ooni/probe-cli/pull/373#discussion_r649820724
Co-authored-by: Arturo Filastò <arturo@openobservatory.org>
2021-06-14 10:20:54 +02:00
|
|
|
MockDialContext: func(ctx context.Context, network string, address string) (net.Conn, error) {
|
|
|
|
// We cancel the context before returning the error, which makes
|
|
|
|
// the context cancellation happen before us returning.
|
|
|
|
cancel()
|
|
|
|
conn, err := net.Dial(network, address)
|
|
|
|
if err != nil {
|
|
|
|
return nil, err
|
|
|
|
}
|
|
|
|
return &obfs4connwrapper{
|
|
|
|
Conn: conn,
|
|
|
|
called: called,
|
|
|
|
}, nil
|
|
|
|
},
|
|
|
|
}
|
|
|
|
cd, err := o4d.newCancellableDialer()
|
|
|
|
if err != nil {
|
|
|
|
t.Fatal(err)
|
|
|
|
}
|
|
|
|
conn, err := cd.dial(ctx, "tcp", o4d.Address)
|
|
|
|
if !errors.Is(err, context.Canceled) {
|
|
|
|
t.Fatal("not the error we expected", err)
|
|
|
|
}
|
|
|
|
if conn != nil {
|
|
|
|
t.Fatal("expected nil conn here")
|
|
|
|
}
|
|
|
|
// The point of returning early when the context expires is
|
|
|
|
// to NOT wait for the background goroutine to terminate, but
|
|
|
|
// here we wanna observe whether it terminates and whether
|
|
|
|
// it calls close. Hence, well, we need to wait :^).
|
|
|
|
<-cd.done
|
|
|
|
if called.Load() != 1 {
|
|
|
|
t.Fatal("the goroutine did not call close")
|
|
|
|
}
|
|
|
|
}
|