ooni-probe-cli/internal/tunnel/config.go

210 lines
6.9 KiB
Go
Raw Permalink Normal View History

package tunnel
import (
"context"
feat(torsf): collect tor logs, select rendezvous method, count bytes (#683) This diff contains significant improvements over the previous implementation of the torsf experiment. We add support for configuring different rendezvous methods after the convo at https://github.com/ooni/probe/issues/2004. In doing that, I've tried to use a terminology that is consistent with the names being actually used by tor developers. In terms of what to do next, this diff basically instruments torsf to always rendezvous using domain fronting. Yet, it's also possible to change the rendezvous method from the command line, when using miniooni, which allows to experiment a bit more. In the same vein, by default we use a persistent tor datadir, but it's also possible to use a temporary datadir using the cmdline. Here's how a generic invocation of `torsf` looks like: ```bash ./miniooni -O DisablePersistentDatadir=true \ -O RendezvousMethod=amp \ -O DisableProgress=true \ torsf ``` (The default is `DisablePersistentDatadir=false` and `RendezvousMethod=domain_fronting`.) With this implementation, we can start measuring whether snowflake and tor together can boostrap, which seems the most important thing to focus on at the beginning. Understanding why the bootstrap most often does not converge with a temporary datadir on Android devices remains instead an open problem for now. (I'll also update the relevant issues or create new issues after commit this.) We also address some methodology improvements that were proposed in https://github.com/ooni/probe/issues/1686. Namely: 1. we record the tor version; 2. we include the bootstrap percentage by reading the logs; 3. we set the anomaly key correctly; 4. we measure the bytes send and received (by `tor` not by `snowflake`, since doing it for snowflake seems more complex at this stage). What remains to be done is the possibility of including Snowflake events into the measurement, which is not possible until the new improvements at common/event in snowflake.git are included into a tagged version of snowflake itself. (I'll make sure to mention this aspect to @cohosh in https://github.com/ooni/probe/issues/2004.)
2022-02-07 17:05:36 +01:00
"errors"
"net"
"os"
"github.com/armon/go-socks5"
"github.com/cretz/bine/control"
"github.com/cretz/bine/tor"
"github.com/ooni/probe-cli/v3/internal/model"
"github.com/ooni/probe-cli/v3/internal/ptx"
"golang.org/x/sys/execabs"
)
// Config contains the configuration for creating a Tunnel instance. You need
// to fill all the mandatory fields. You SHOULD NOT modify the content of this
// structure while in use, because that may lead to data races.
type Config struct {
// Name is the MANDATORY name of the tunnel. We support
// "tor", "psiphon", and "fake" tunnels. You SHOULD
// use "fake" tunnels only for testing: they don't provide
// any real tunneling, just a socks5 proxy.
Name string
// Session is the MANDATORY measurement session, or a suitable
// mock of the required functionality. That is, the possibility
// of obtaining a valid psiphon configuration.
Session Session
// SnowflakeRendezvous is the OPTIONAL rendezvous
// method for snowflake
SnowflakeRendezvous string
// TunnelDir is the MANDATORY directory in which the tunnel SHOULD
// store its state, if any. If this field is empty, the
// Start function fails with ErrEmptyTunnelDir.
TunnelDir string
// Logger is the optional logger to use. If empty we use a default
// implementation that does not emit any output.
Logger model.Logger
// TorArgs contains the optional arguments that you want us to pass
// to the tor binary when invoking it. By default we do not
// pass any extra argument. This flag might be useful to
// configure pluggable transports.
TorArgs []string
// TorBinary is the optional path of the TorBinary we SHOULD be
// executing. When not set, we execute `tor`.
TorBinary string
// testExecabsLookPath allows us to mock exeabs.LookPath
testExecabsLookPath func(name string) (string, error)
// testMkdirAll allows us to mock os.MkdirAll in testing code.
testMkdirAll func(path string, perm os.FileMode) error
// testNetListen allows us to mock net.Listen in testing code.
testNetListen func(network string, address string) (net.Listener, error)
// testSfListenSocks is OPTIONAL and allows to override the
// ListenSocks field of a ptx.Listener.
testSfListenSocks func(network string, laddr string) (ptx.SocksListener, error)
// testSfNewPTXListener is OPTIONAL and allows us to wrap the
// constructed ptx.Listener for testing purposes.
testSfWrapPTXListener func(torsfPTXListener) torsfPTXListener
// testSfTorStart is OPTIONAL and allows us to override the
// call to torStart inside the torsf tunnel.
testSfTorStart func(ctx context.Context, config *Config) (Tunnel, DebugInfo, error)
// testSocks5New allows us to mock socks5.New in testing code.
testSocks5New func(conf *socks5.Config) (*socks5.Server, error)
// testTorStart allows us to mock tor.Start.
testTorStart func(ctx context.Context, conf *tor.StartConf) (*tor.Tor, error)
feat(torsf): collect tor logs, select rendezvous method, count bytes (#683) This diff contains significant improvements over the previous implementation of the torsf experiment. We add support for configuring different rendezvous methods after the convo at https://github.com/ooni/probe/issues/2004. In doing that, I've tried to use a terminology that is consistent with the names being actually used by tor developers. In terms of what to do next, this diff basically instruments torsf to always rendezvous using domain fronting. Yet, it's also possible to change the rendezvous method from the command line, when using miniooni, which allows to experiment a bit more. In the same vein, by default we use a persistent tor datadir, but it's also possible to use a temporary datadir using the cmdline. Here's how a generic invocation of `torsf` looks like: ```bash ./miniooni -O DisablePersistentDatadir=true \ -O RendezvousMethod=amp \ -O DisableProgress=true \ torsf ``` (The default is `DisablePersistentDatadir=false` and `RendezvousMethod=domain_fronting`.) With this implementation, we can start measuring whether snowflake and tor together can boostrap, which seems the most important thing to focus on at the beginning. Understanding why the bootstrap most often does not converge with a temporary datadir on Android devices remains instead an open problem for now. (I'll also update the relevant issues or create new issues after commit this.) We also address some methodology improvements that were proposed in https://github.com/ooni/probe/issues/1686. Namely: 1. we record the tor version; 2. we include the bootstrap percentage by reading the logs; 3. we set the anomaly key correctly; 4. we measure the bytes send and received (by `tor` not by `snowflake`, since doing it for snowflake seems more complex at this stage). What remains to be done is the possibility of including Snowflake events into the measurement, which is not possible until the new improvements at common/event in snowflake.git are included into a tagged version of snowflake itself. (I'll make sure to mention this aspect to @cohosh in https://github.com/ooni/probe/issues/2004.)
2022-02-07 17:05:36 +01:00
// testTorProtocolInfo allows us to mock getting protocol info.
testTorProtocolInfo func(tor *tor.Tor) (*control.ProtocolInfo, error)
// testTorEnableNetwork allows us to fake a failure when
// telling to the tor daemon to enable the network.
testTorEnableNetwork func(ctx context.Context, tor *tor.Tor, wait bool) error
// testTorGetInfo allows us to fake a failure when
// getting info from the tor control port.
testTorGetInfo func(ctrl *control.Conn, keys ...string) ([]*control.KeyVal, error)
}
// snowflakeRendezvousMethod returns the rendezvous method that snowflake should use
func (c *Config) snowflakeRendezvousMethod() string {
if c.SnowflakeRendezvous != "" {
return c.SnowflakeRendezvous
}
return "domain_fronting"
}
// logger returns the logger to use.
func (c *Config) logger() model.Logger {
if c.Logger != nil {
return c.Logger
}
return model.DiscardLogger
}
// execabsLookPath calls either testExeabsLookPath or execabs.LookPath
func (c *Config) execabsLookPath(name string) (string, error) {
if c.testExecabsLookPath != nil {
return c.testExecabsLookPath(name)
}
return execabs.LookPath(name)
}
// mkdirAll calls either testMkdirAll or os.MkdirAll.
func (c *Config) mkdirAll(path string, perm os.FileMode) error {
if c.testMkdirAll != nil {
return c.testMkdirAll(path, perm)
}
return os.MkdirAll(path, perm)
}
// netListen calls either testNetListen or net.Listen.
func (c *Config) netListen(network string, address string) (net.Listener, error) {
if c.testNetListen != nil {
return c.testNetListen(network, address)
}
return net.Listen(network, address)
}
// socks5New calls either testSocks5New or socks5.New
func (c *Config) socks5New(conf *socks5.Config) (*socks5.Server, error) {
if c.testSocks5New != nil {
return c.testSocks5New(conf)
}
return socks5.New(conf)
}
// ooniTorBinaryEnv is the name of the environment variable
// we're using to get the path to the tor binary when we are
// being run by the ooni/probe-desktop application.
const ooniTorBinaryEnv = "OONI_TOR_BINARY"
// torBinary returns the tor binary path.
//
// Here's is the algorithm:
//
// 1. if c.TorBinary is set, we use its value;
//
// 2. if os.Getenv("OONI_TOR_BINARY") is set, we use its value;
//
// 3. otherwise, we return "tor".
//
// Implementation note: in cases 1 and 3 we use execabs.LookPath
// to guarantee we're not going to execute a binary outside of the
// PATH (see https://blog.golang.org/path-security for more info
// on how this bug could affect Windows). In case 2, we're instead
// just going to trust the binary set by the probe-desktop app.
func (c *Config) torBinary() (string, error) {
if c.TorBinary != "" {
return c.execabsLookPath(c.TorBinary)
}
if binary := os.Getenv(ooniTorBinaryEnv); binary != "" {
return binary, nil
}
return c.execabsLookPath("tor")
}
// torStart calls either testTorStart or tor.Start.
func (c *Config) torStart(ctx context.Context, conf *tor.StartConf) (*tor.Tor, error) {
if c.testTorStart != nil {
return c.testTorStart(ctx, conf)
}
return tor.Start(ctx, conf)
}
feat(torsf): collect tor logs, select rendezvous method, count bytes (#683) This diff contains significant improvements over the previous implementation of the torsf experiment. We add support for configuring different rendezvous methods after the convo at https://github.com/ooni/probe/issues/2004. In doing that, I've tried to use a terminology that is consistent with the names being actually used by tor developers. In terms of what to do next, this diff basically instruments torsf to always rendezvous using domain fronting. Yet, it's also possible to change the rendezvous method from the command line, when using miniooni, which allows to experiment a bit more. In the same vein, by default we use a persistent tor datadir, but it's also possible to use a temporary datadir using the cmdline. Here's how a generic invocation of `torsf` looks like: ```bash ./miniooni -O DisablePersistentDatadir=true \ -O RendezvousMethod=amp \ -O DisableProgress=true \ torsf ``` (The default is `DisablePersistentDatadir=false` and `RendezvousMethod=domain_fronting`.) With this implementation, we can start measuring whether snowflake and tor together can boostrap, which seems the most important thing to focus on at the beginning. Understanding why the bootstrap most often does not converge with a temporary datadir on Android devices remains instead an open problem for now. (I'll also update the relevant issues or create new issues after commit this.) We also address some methodology improvements that were proposed in https://github.com/ooni/probe/issues/1686. Namely: 1. we record the tor version; 2. we include the bootstrap percentage by reading the logs; 3. we set the anomaly key correctly; 4. we measure the bytes send and received (by `tor` not by `snowflake`, since doing it for snowflake seems more complex at this stage). What remains to be done is the possibility of including Snowflake events into the measurement, which is not possible until the new improvements at common/event in snowflake.git are included into a tagged version of snowflake itself. (I'll make sure to mention this aspect to @cohosh in https://github.com/ooni/probe/issues/2004.)
2022-02-07 17:05:36 +01:00
// errNoTorControl indicate you passed us a tor with a nil control field.
var errNoTorControl = errors.New("tunnel: no tor control")
// torProtocolInfo calls either testTorProtocolInfo or the
// proper function to get back protocol information.
func (c *Config) torProtocolInfo(tor *tor.Tor) (*control.ProtocolInfo, error) {
if c.testTorProtocolInfo != nil {
return c.testTorProtocolInfo(tor)
}
if tor.Control == nil {
return nil, errNoTorControl
}
return tor.Control.ProtocolInfo()
}
// torEnableNetwork calls either testTorEnableNetwork or tor.EnableNetwork.
func (c *Config) torEnableNetwork(ctx context.Context, tor *tor.Tor, wait bool) error {
if c.testTorEnableNetwork != nil {
return c.testTorEnableNetwork(ctx, tor, wait)
}
return tor.EnableNetwork(ctx, wait)
}
// torGetInfo calls either testTorGetInfo or ctrl.GetInfo.
func (c *Config) torGetInfo(ctrl *control.Conn, keys ...string) ([]*control.KeyVal, error) {
if c.testTorGetInfo != nil {
return c.testTorGetInfo(ctrl, keys...)
}
return ctrl.GetInfo(keys...)
}