refactor: start pivoting netx (#396)
What do I mean by pivoting? Netx is currently organized by row:
```
| dialer | quicdialer | resolver | ...
saving | | | | ...
errorwrapping | | | | ...
logging | | | | ...
mocking/sys | | | | ...
```
Every row needs to implement saving, errorwrapping, logging, mocking (or
adapting to the system or to some underlying library).
This causes cross package dependencies and, in turn, complexity. For
example, we need the `trace` package for supporting saving.
And `dialer`, `quickdialer`, et al. need to depend on such a package.
The same goes for errorwrapping.
This arrangement further complicates testing. For example, I am
currently working on https://github.com/ooni/probe/issues/1505 and
I realize it need to repeat integration tests in multiple places.
Let's say instead we pivot the above matrix as follows:
```
| saving | errorwrapping | logging | ...
dialer | | | | ...
quicdialer | | | | ...
logging | | | | ...
mocking/sys | | | | ...
...
```
In this way, now every row contains everything related to a specific
action to perform. We can now share code without relying on extra
support packages. What's more, we can write tests and, judding from
the way in which things are made, it seems we only need integration
testing in `errorwrapping` because it's where data quality matters
whereas, in all other cases, unit testing is fine.
I am going, therefore, to proceed with these changes and "pivot"
`netx`. Hopefully, it won't be too painful.
2021-06-23 15:53:12 +02:00
|
|
|
package netxlite
|
|
|
|
|
2022-05-15 19:25:27 +02:00
|
|
|
//
|
|
|
|
// Code for dialing TCP or UDP net.Conn-like connections
|
|
|
|
//
|
|
|
|
|
refactor: start pivoting netx (#396)
What do I mean by pivoting? Netx is currently organized by row:
```
| dialer | quicdialer | resolver | ...
saving | | | | ...
errorwrapping | | | | ...
logging | | | | ...
mocking/sys | | | | ...
```
Every row needs to implement saving, errorwrapping, logging, mocking (or
adapting to the system or to some underlying library).
This causes cross package dependencies and, in turn, complexity. For
example, we need the `trace` package for supporting saving.
And `dialer`, `quickdialer`, et al. need to depend on such a package.
The same goes for errorwrapping.
This arrangement further complicates testing. For example, I am
currently working on https://github.com/ooni/probe/issues/1505 and
I realize it need to repeat integration tests in multiple places.
Let's say instead we pivot the above matrix as follows:
```
| saving | errorwrapping | logging | ...
dialer | | | | ...
quicdialer | | | | ...
logging | | | | ...
mocking/sys | | | | ...
...
```
In this way, now every row contains everything related to a specific
action to perform. We can now share code without relying on extra
support packages. What's more, we can write tests and, judding from
the way in which things are made, it seems we only need integration
testing in `errorwrapping` because it's where data quality matters
whereas, in all other cases, unit testing is fine.
I am going, therefore, to proceed with these changes and "pivot"
`netx`. Hopefully, it won't be too painful.
2021-06-23 15:53:12 +02:00
|
|
|
import (
|
|
|
|
"context"
|
2021-09-06 14:12:30 +02:00
|
|
|
"errors"
|
refactor: start pivoting netx (#396)
What do I mean by pivoting? Netx is currently organized by row:
```
| dialer | quicdialer | resolver | ...
saving | | | | ...
errorwrapping | | | | ...
logging | | | | ...
mocking/sys | | | | ...
```
Every row needs to implement saving, errorwrapping, logging, mocking (or
adapting to the system or to some underlying library).
This causes cross package dependencies and, in turn, complexity. For
example, we need the `trace` package for supporting saving.
And `dialer`, `quickdialer`, et al. need to depend on such a package.
The same goes for errorwrapping.
This arrangement further complicates testing. For example, I am
currently working on https://github.com/ooni/probe/issues/1505 and
I realize it need to repeat integration tests in multiple places.
Let's say instead we pivot the above matrix as follows:
```
| saving | errorwrapping | logging | ...
dialer | | | | ...
quicdialer | | | | ...
logging | | | | ...
mocking/sys | | | | ...
...
```
In this way, now every row contains everything related to a specific
action to perform. We can now share code without relying on extra
support packages. What's more, we can write tests and, judding from
the way in which things are made, it seems we only need integration
testing in `errorwrapping` because it's where data quality matters
whereas, in all other cases, unit testing is fine.
I am going, therefore, to proceed with these changes and "pivot"
`netx`. Hopefully, it won't be too painful.
2021-06-23 15:53:12 +02:00
|
|
|
"net"
|
2021-09-06 14:12:30 +02:00
|
|
|
"sync"
|
refactor: start pivoting netx (#396)
What do I mean by pivoting? Netx is currently organized by row:
```
| dialer | quicdialer | resolver | ...
saving | | | | ...
errorwrapping | | | | ...
logging | | | | ...
mocking/sys | | | | ...
```
Every row needs to implement saving, errorwrapping, logging, mocking (or
adapting to the system or to some underlying library).
This causes cross package dependencies and, in turn, complexity. For
example, we need the `trace` package for supporting saving.
And `dialer`, `quickdialer`, et al. need to depend on such a package.
The same goes for errorwrapping.
This arrangement further complicates testing. For example, I am
currently working on https://github.com/ooni/probe/issues/1505 and
I realize it need to repeat integration tests in multiple places.
Let's say instead we pivot the above matrix as follows:
```
| saving | errorwrapping | logging | ...
dialer | | | | ...
quicdialer | | | | ...
logging | | | | ...
mocking/sys | | | | ...
...
```
In this way, now every row contains everything related to a specific
action to perform. We can now share code without relying on extra
support packages. What's more, we can write tests and, judding from
the way in which things are made, it seems we only need integration
testing in `errorwrapping` because it's where data quality matters
whereas, in all other cases, unit testing is fine.
I am going, therefore, to proceed with these changes and "pivot"
`netx`. Hopefully, it won't be too painful.
2021-06-23 15:53:12 +02:00
|
|
|
"time"
|
2021-09-05 19:55:28 +02:00
|
|
|
|
2022-01-03 13:53:23 +01:00
|
|
|
"github.com/ooni/probe-cli/v3/internal/model"
|
|
|
|
)
|
refactor: start pivoting netx (#396)
What do I mean by pivoting? Netx is currently organized by row:
```
| dialer | quicdialer | resolver | ...
saving | | | | ...
errorwrapping | | | | ...
logging | | | | ...
mocking/sys | | | | ...
```
Every row needs to implement saving, errorwrapping, logging, mocking (or
adapting to the system or to some underlying library).
This causes cross package dependencies and, in turn, complexity. For
example, we need the `trace` package for supporting saving.
And `dialer`, `quickdialer`, et al. need to depend on such a package.
The same goes for errorwrapping.
This arrangement further complicates testing. For example, I am
currently working on https://github.com/ooni/probe/issues/1505 and
I realize it need to repeat integration tests in multiple places.
Let's say instead we pivot the above matrix as follows:
```
| saving | errorwrapping | logging | ...
dialer | | | | ...
quicdialer | | | | ...
logging | | | | ...
mocking/sys | | | | ...
...
```
In this way, now every row contains everything related to a specific
action to perform. We can now share code without relying on extra
support packages. What's more, we can write tests and, judding from
the way in which things are made, it seems we only need integration
testing in `errorwrapping` because it's where data quality matters
whereas, in all other cases, unit testing is fine.
I am going, therefore, to proceed with these changes and "pivot"
`netx`. Hopefully, it won't be too painful.
2021-06-23 15:53:12 +02:00
|
|
|
|
2021-09-29 20:21:25 +02:00
|
|
|
// NewDialerWithResolver calls WrapDialer for the stdlib dialer.
|
2022-01-03 13:53:23 +01:00
|
|
|
func NewDialerWithResolver(logger model.DebugLogger, resolver model.Resolver) model.Dialer {
|
2021-09-27 12:00:43 +02:00
|
|
|
return WrapDialer(logger, resolver, &dialerSystem{})
|
|
|
|
}
|
|
|
|
|
|
|
|
// WrapDialer creates a new Dialer that wraps the given
|
|
|
|
// Dialer. The returned Dialer has the following properties:
|
2021-09-08 00:00:53 +02:00
|
|
|
//
|
2021-09-08 21:19:51 +02:00
|
|
|
// 1. logs events using the given logger;
|
2021-09-08 00:00:53 +02:00
|
|
|
//
|
2021-09-08 21:19:51 +02:00
|
|
|
// 2. resolves domain names using the givern resolver;
|
2021-09-08 00:00:53 +02:00
|
|
|
//
|
2021-09-29 20:21:25 +02:00
|
|
|
// 3. when the resolver is not a "null" resolver,
|
|
|
|
// each available enpoint is tried
|
2021-09-08 21:19:51 +02:00
|
|
|
// sequentially. On error, the code will return what it believes
|
|
|
|
// to be the most representative error in the pack. Most often,
|
2021-09-29 20:21:25 +02:00
|
|
|
// the first error that occurred. Choosing the
|
2021-09-08 21:19:51 +02:00
|
|
|
// error to return using this logic is a QUIRK that we owe
|
|
|
|
// to the original implementation of netx. We cannot change
|
2021-09-29 20:21:25 +02:00
|
|
|
// this behavior until we refactor legacy code using it.
|
2021-09-08 00:00:53 +02:00
|
|
|
//
|
2021-09-08 21:19:51 +02:00
|
|
|
// Removing this quirk from the codebase is documented as
|
|
|
|
// TODO(https://github.com/ooni/probe/issues/1779).
|
|
|
|
//
|
|
|
|
// 4. wraps errors;
|
|
|
|
//
|
2021-09-08 22:48:10 +02:00
|
|
|
// 5. has a configured connect timeout;
|
|
|
|
//
|
|
|
|
// 6. if a dialer wraps a resolver, the dialer will forward
|
|
|
|
// the CloseIdleConnection call to its resolver (which is
|
|
|
|
// instrumental to manage a DoH resolver connections properly).
|
2021-09-29 20:21:25 +02:00
|
|
|
//
|
|
|
|
// In general, do not use WrapDialer directly but try to use
|
|
|
|
// more high-level factories, e.g., NewDialerWithResolver.
|
2022-01-03 13:53:23 +01:00
|
|
|
func WrapDialer(logger model.DebugLogger, resolver model.Resolver, dialer model.Dialer) model.Dialer {
|
2021-09-05 20:41:46 +02:00
|
|
|
return &dialerLogger{
|
|
|
|
Dialer: &dialerResolver{
|
|
|
|
Dialer: &dialerLogger{
|
2021-09-07 19:56:42 +02:00
|
|
|
Dialer: &dialerErrWrapper{
|
2021-09-27 12:00:43 +02:00
|
|
|
Dialer: dialer,
|
2021-09-07 19:56:42 +02:00
|
|
|
},
|
2022-01-03 13:53:23 +01:00
|
|
|
DebugLogger: logger,
|
2021-09-06 21:34:14 +02:00
|
|
|
operationSuffix: "_address",
|
2021-09-05 20:41:46 +02:00
|
|
|
},
|
|
|
|
Resolver: resolver,
|
|
|
|
},
|
2022-01-03 13:53:23 +01:00
|
|
|
DebugLogger: logger,
|
2021-09-05 20:41:46 +02:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2021-09-29 20:21:25 +02:00
|
|
|
// NewDialerWithoutResolver calls NewDialerWithResolver with a "null" resolver.
|
|
|
|
//
|
|
|
|
// The returned dialer fails with ErrNoResolver if passed a domain name.
|
2022-01-03 13:53:23 +01:00
|
|
|
func NewDialerWithoutResolver(logger model.DebugLogger) model.Dialer {
|
2021-09-05 20:41:46 +02:00
|
|
|
return NewDialerWithResolver(logger, &nullResolver{})
|
|
|
|
}
|
|
|
|
|
2021-09-08 21:19:51 +02:00
|
|
|
// dialerSystem uses system facilities to perform domain name
|
|
|
|
// resolution and guarantees we have a dialer timeout.
|
2021-09-08 00:00:53 +02:00
|
|
|
type dialerSystem struct {
|
|
|
|
// timeout is the OPTIONAL timeout used for testing.
|
|
|
|
timeout time.Duration
|
refactor: start pivoting netx (#396)
What do I mean by pivoting? Netx is currently organized by row:
```
| dialer | quicdialer | resolver | ...
saving | | | | ...
errorwrapping | | | | ...
logging | | | | ...
mocking/sys | | | | ...
```
Every row needs to implement saving, errorwrapping, logging, mocking (or
adapting to the system or to some underlying library).
This causes cross package dependencies and, in turn, complexity. For
example, we need the `trace` package for supporting saving.
And `dialer`, `quickdialer`, et al. need to depend on such a package.
The same goes for errorwrapping.
This arrangement further complicates testing. For example, I am
currently working on https://github.com/ooni/probe/issues/1505 and
I realize it need to repeat integration tests in multiple places.
Let's say instead we pivot the above matrix as follows:
```
| saving | errorwrapping | logging | ...
dialer | | | | ...
quicdialer | | | | ...
logging | | | | ...
mocking/sys | | | | ...
...
```
In this way, now every row contains everything related to a specific
action to perform. We can now share code without relying on extra
support packages. What's more, we can write tests and, judding from
the way in which things are made, it seems we only need integration
testing in `errorwrapping` because it's where data quality matters
whereas, in all other cases, unit testing is fine.
I am going, therefore, to proceed with these changes and "pivot"
`netx`. Hopefully, it won't be too painful.
2021-06-23 15:53:12 +02:00
|
|
|
}
|
|
|
|
|
2022-01-03 13:53:23 +01:00
|
|
|
var _ model.Dialer = &dialerSystem{}
|
2021-09-08 21:19:51 +02:00
|
|
|
|
|
|
|
const dialerDefaultTimeout = 15 * time.Second
|
|
|
|
|
2022-01-03 13:53:23 +01:00
|
|
|
func (d *dialerSystem) newUnderlyingDialer() model.SimpleDialer {
|
2021-09-08 00:00:53 +02:00
|
|
|
t := d.timeout
|
|
|
|
if t <= 0 {
|
2021-09-08 21:19:51 +02:00
|
|
|
t = dialerDefaultTimeout
|
2021-09-08 00:00:53 +02:00
|
|
|
}
|
2022-01-03 13:53:23 +01:00
|
|
|
return TProxy.NewSimpleDialer(t)
|
2021-09-08 00:00:53 +02:00
|
|
|
}
|
2021-09-05 19:55:28 +02:00
|
|
|
|
|
|
|
func (d *dialerSystem) DialContext(ctx context.Context, network, address string) (net.Conn, error) {
|
2021-09-08 00:00:53 +02:00
|
|
|
return d.newUnderlyingDialer().DialContext(ctx, network, address)
|
2021-09-05 19:55:28 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
func (d *dialerSystem) CloseIdleConnections() {
|
2021-09-08 21:19:51 +02:00
|
|
|
// nothing to do here
|
2021-09-05 19:55:28 +02:00
|
|
|
}
|
|
|
|
|
2021-09-08 21:19:51 +02:00
|
|
|
// dialerResolver combines dialing with domain name resolution.
|
2021-09-05 14:49:38 +02:00
|
|
|
type dialerResolver struct {
|
2022-05-15 19:25:27 +02:00
|
|
|
Dialer model.Dialer
|
|
|
|
Resolver model.Resolver
|
refactor: start pivoting netx (#396)
What do I mean by pivoting? Netx is currently organized by row:
```
| dialer | quicdialer | resolver | ...
saving | | | | ...
errorwrapping | | | | ...
logging | | | | ...
mocking/sys | | | | ...
```
Every row needs to implement saving, errorwrapping, logging, mocking (or
adapting to the system or to some underlying library).
This causes cross package dependencies and, in turn, complexity. For
example, we need the `trace` package for supporting saving.
And `dialer`, `quickdialer`, et al. need to depend on such a package.
The same goes for errorwrapping.
This arrangement further complicates testing. For example, I am
currently working on https://github.com/ooni/probe/issues/1505 and
I realize it need to repeat integration tests in multiple places.
Let's say instead we pivot the above matrix as follows:
```
| saving | errorwrapping | logging | ...
dialer | | | | ...
quicdialer | | | | ...
logging | | | | ...
mocking/sys | | | | ...
...
```
In this way, now every row contains everything related to a specific
action to perform. We can now share code without relying on extra
support packages. What's more, we can write tests and, judding from
the way in which things are made, it seems we only need integration
testing in `errorwrapping` because it's where data quality matters
whereas, in all other cases, unit testing is fine.
I am going, therefore, to proceed with these changes and "pivot"
`netx`. Hopefully, it won't be too painful.
2021-06-23 15:53:12 +02:00
|
|
|
}
|
|
|
|
|
2022-01-03 13:53:23 +01:00
|
|
|
var _ model.Dialer = &dialerResolver{}
|
refactor: start pivoting netx (#396)
What do I mean by pivoting? Netx is currently organized by row:
```
| dialer | quicdialer | resolver | ...
saving | | | | ...
errorwrapping | | | | ...
logging | | | | ...
mocking/sys | | | | ...
```
Every row needs to implement saving, errorwrapping, logging, mocking (or
adapting to the system or to some underlying library).
This causes cross package dependencies and, in turn, complexity. For
example, we need the `trace` package for supporting saving.
And `dialer`, `quickdialer`, et al. need to depend on such a package.
The same goes for errorwrapping.
This arrangement further complicates testing. For example, I am
currently working on https://github.com/ooni/probe/issues/1505 and
I realize it need to repeat integration tests in multiple places.
Let's say instead we pivot the above matrix as follows:
```
| saving | errorwrapping | logging | ...
dialer | | | | ...
quicdialer | | | | ...
logging | | | | ...
mocking/sys | | | | ...
...
```
In this way, now every row contains everything related to a specific
action to perform. We can now share code without relying on extra
support packages. What's more, we can write tests and, judding from
the way in which things are made, it seems we only need integration
testing in `errorwrapping` because it's where data quality matters
whereas, in all other cases, unit testing is fine.
I am going, therefore, to proceed with these changes and "pivot"
`netx`. Hopefully, it won't be too painful.
2021-06-23 15:53:12 +02:00
|
|
|
|
2021-09-05 14:49:38 +02:00
|
|
|
func (d *dialerResolver) DialContext(ctx context.Context, network, address string) (net.Conn, error) {
|
2021-09-08 21:19:51 +02:00
|
|
|
// QUIRK: this routine and the related routines in quirks.go cannot
|
|
|
|
// be changed easily until we use events tracing to measure.
|
|
|
|
//
|
|
|
|
// Reference issue: TODO(https://github.com/ooni/probe/issues/1779).
|
refactor: start pivoting netx (#396)
What do I mean by pivoting? Netx is currently organized by row:
```
| dialer | quicdialer | resolver | ...
saving | | | | ...
errorwrapping | | | | ...
logging | | | | ...
mocking/sys | | | | ...
```
Every row needs to implement saving, errorwrapping, logging, mocking (or
adapting to the system or to some underlying library).
This causes cross package dependencies and, in turn, complexity. For
example, we need the `trace` package for supporting saving.
And `dialer`, `quickdialer`, et al. need to depend on such a package.
The same goes for errorwrapping.
This arrangement further complicates testing. For example, I am
currently working on https://github.com/ooni/probe/issues/1505 and
I realize it need to repeat integration tests in multiple places.
Let's say instead we pivot the above matrix as follows:
```
| saving | errorwrapping | logging | ...
dialer | | | | ...
quicdialer | | | | ...
logging | | | | ...
mocking/sys | | | | ...
...
```
In this way, now every row contains everything related to a specific
action to perform. We can now share code without relying on extra
support packages. What's more, we can write tests and, judding from
the way in which things are made, it seems we only need integration
testing in `errorwrapping` because it's where data quality matters
whereas, in all other cases, unit testing is fine.
I am going, therefore, to proceed with these changes and "pivot"
`netx`. Hopefully, it won't be too painful.
2021-06-23 15:53:12 +02:00
|
|
|
onlyhost, onlyport, err := net.SplitHostPort(address)
|
|
|
|
if err != nil {
|
|
|
|
return nil, err
|
|
|
|
}
|
2021-06-23 17:00:44 +02:00
|
|
|
addrs, err := d.lookupHost(ctx, onlyhost)
|
refactor: start pivoting netx (#396)
What do I mean by pivoting? Netx is currently organized by row:
```
| dialer | quicdialer | resolver | ...
saving | | | | ...
errorwrapping | | | | ...
logging | | | | ...
mocking/sys | | | | ...
```
Every row needs to implement saving, errorwrapping, logging, mocking (or
adapting to the system or to some underlying library).
This causes cross package dependencies and, in turn, complexity. For
example, we need the `trace` package for supporting saving.
And `dialer`, `quickdialer`, et al. need to depend on such a package.
The same goes for errorwrapping.
This arrangement further complicates testing. For example, I am
currently working on https://github.com/ooni/probe/issues/1505 and
I realize it need to repeat integration tests in multiple places.
Let's say instead we pivot the above matrix as follows:
```
| saving | errorwrapping | logging | ...
dialer | | | | ...
quicdialer | | | | ...
logging | | | | ...
mocking/sys | | | | ...
...
```
In this way, now every row contains everything related to a specific
action to perform. We can now share code without relying on extra
support packages. What's more, we can write tests and, judding from
the way in which things are made, it seems we only need integration
testing in `errorwrapping` because it's where data quality matters
whereas, in all other cases, unit testing is fine.
I am going, therefore, to proceed with these changes and "pivot"
`netx`. Hopefully, it won't be too painful.
2021-06-23 15:53:12 +02:00
|
|
|
if err != nil {
|
|
|
|
return nil, err
|
|
|
|
}
|
2021-09-06 20:17:45 +02:00
|
|
|
addrs = quirkSortIPAddrs(addrs)
|
refactor: start pivoting netx (#396)
What do I mean by pivoting? Netx is currently organized by row:
```
| dialer | quicdialer | resolver | ...
saving | | | | ...
errorwrapping | | | | ...
logging | | | | ...
mocking/sys | | | | ...
```
Every row needs to implement saving, errorwrapping, logging, mocking (or
adapting to the system or to some underlying library).
This causes cross package dependencies and, in turn, complexity. For
example, we need the `trace` package for supporting saving.
And `dialer`, `quickdialer`, et al. need to depend on such a package.
The same goes for errorwrapping.
This arrangement further complicates testing. For example, I am
currently working on https://github.com/ooni/probe/issues/1505 and
I realize it need to repeat integration tests in multiple places.
Let's say instead we pivot the above matrix as follows:
```
| saving | errorwrapping | logging | ...
dialer | | | | ...
quicdialer | | | | ...
logging | | | | ...
mocking/sys | | | | ...
...
```
In this way, now every row contains everything related to a specific
action to perform. We can now share code without relying on extra
support packages. What's more, we can write tests and, judding from
the way in which things are made, it seems we only need integration
testing in `errorwrapping` because it's where data quality matters
whereas, in all other cases, unit testing is fine.
I am going, therefore, to proceed with these changes and "pivot"
`netx`. Hopefully, it won't be too painful.
2021-06-23 15:53:12 +02:00
|
|
|
var errorslist []error
|
|
|
|
for _, addr := range addrs {
|
|
|
|
target := net.JoinHostPort(addr, onlyport)
|
|
|
|
conn, err := d.Dialer.DialContext(ctx, network, target)
|
|
|
|
if err == nil {
|
|
|
|
return conn, nil
|
|
|
|
}
|
|
|
|
errorslist = append(errorslist, err)
|
|
|
|
}
|
2021-09-06 20:17:45 +02:00
|
|
|
return nil, quirkReduceErrors(errorslist)
|
refactor: start pivoting netx (#396)
What do I mean by pivoting? Netx is currently organized by row:
```
| dialer | quicdialer | resolver | ...
saving | | | | ...
errorwrapping | | | | ...
logging | | | | ...
mocking/sys | | | | ...
```
Every row needs to implement saving, errorwrapping, logging, mocking (or
adapting to the system or to some underlying library).
This causes cross package dependencies and, in turn, complexity. For
example, we need the `trace` package for supporting saving.
And `dialer`, `quickdialer`, et al. need to depend on such a package.
The same goes for errorwrapping.
This arrangement further complicates testing. For example, I am
currently working on https://github.com/ooni/probe/issues/1505 and
I realize it need to repeat integration tests in multiple places.
Let's say instead we pivot the above matrix as follows:
```
| saving | errorwrapping | logging | ...
dialer | | | | ...
quicdialer | | | | ...
logging | | | | ...
mocking/sys | | | | ...
...
```
In this way, now every row contains everything related to a specific
action to perform. We can now share code without relying on extra
support packages. What's more, we can write tests and, judding from
the way in which things are made, it seems we only need integration
testing in `errorwrapping` because it's where data quality matters
whereas, in all other cases, unit testing is fine.
I am going, therefore, to proceed with these changes and "pivot"
`netx`. Hopefully, it won't be too painful.
2021-06-23 15:53:12 +02:00
|
|
|
}
|
|
|
|
|
2021-09-08 21:19:51 +02:00
|
|
|
// lookupHost ensures we correctly handle IP addresses.
|
2021-09-05 14:49:38 +02:00
|
|
|
func (d *dialerResolver) lookupHost(ctx context.Context, hostname string) ([]string, error) {
|
refactor: start pivoting netx (#396)
What do I mean by pivoting? Netx is currently organized by row:
```
| dialer | quicdialer | resolver | ...
saving | | | | ...
errorwrapping | | | | ...
logging | | | | ...
mocking/sys | | | | ...
```
Every row needs to implement saving, errorwrapping, logging, mocking (or
adapting to the system or to some underlying library).
This causes cross package dependencies and, in turn, complexity. For
example, we need the `trace` package for supporting saving.
And `dialer`, `quickdialer`, et al. need to depend on such a package.
The same goes for errorwrapping.
This arrangement further complicates testing. For example, I am
currently working on https://github.com/ooni/probe/issues/1505 and
I realize it need to repeat integration tests in multiple places.
Let's say instead we pivot the above matrix as follows:
```
| saving | errorwrapping | logging | ...
dialer | | | | ...
quicdialer | | | | ...
logging | | | | ...
mocking/sys | | | | ...
...
```
In this way, now every row contains everything related to a specific
action to perform. We can now share code without relying on extra
support packages. What's more, we can write tests and, judding from
the way in which things are made, it seems we only need integration
testing in `errorwrapping` because it's where data quality matters
whereas, in all other cases, unit testing is fine.
I am going, therefore, to proceed with these changes and "pivot"
`netx`. Hopefully, it won't be too painful.
2021-06-23 15:53:12 +02:00
|
|
|
if net.ParseIP(hostname) != nil {
|
|
|
|
return []string{hostname}, nil
|
|
|
|
}
|
|
|
|
return d.Resolver.LookupHost(ctx, hostname)
|
|
|
|
}
|
|
|
|
|
2021-09-05 19:55:28 +02:00
|
|
|
func (d *dialerResolver) CloseIdleConnections() {
|
|
|
|
d.Dialer.CloseIdleConnections()
|
|
|
|
d.Resolver.CloseIdleConnections()
|
|
|
|
}
|
|
|
|
|
2021-09-05 14:49:38 +02:00
|
|
|
// dialerLogger is a Dialer with logging.
|
|
|
|
type dialerLogger struct {
|
2021-06-23 17:00:44 +02:00
|
|
|
// Dialer is the underlying dialer.
|
2022-05-15 19:25:27 +02:00
|
|
|
Dialer model.Dialer
|
2021-06-23 17:00:44 +02:00
|
|
|
|
2022-05-15 19:25:27 +02:00
|
|
|
// DebugLogger is the underlying logger.
|
|
|
|
DebugLogger model.DebugLogger
|
2021-09-06 21:34:14 +02:00
|
|
|
|
|
|
|
// operationSuffix is appended to the operation name.
|
|
|
|
//
|
|
|
|
// We use this suffix to distinguish the output from dialing
|
|
|
|
// with the output from dialing an IP address when we are
|
|
|
|
// using a dialer without resolver, where otherwise both lines
|
|
|
|
// would read something like `dial 8.8.8.8:443...`
|
|
|
|
operationSuffix string
|
refactor: start pivoting netx (#396)
What do I mean by pivoting? Netx is currently organized by row:
```
| dialer | quicdialer | resolver | ...
saving | | | | ...
errorwrapping | | | | ...
logging | | | | ...
mocking/sys | | | | ...
```
Every row needs to implement saving, errorwrapping, logging, mocking (or
adapting to the system or to some underlying library).
This causes cross package dependencies and, in turn, complexity. For
example, we need the `trace` package for supporting saving.
And `dialer`, `quickdialer`, et al. need to depend on such a package.
The same goes for errorwrapping.
This arrangement further complicates testing. For example, I am
currently working on https://github.com/ooni/probe/issues/1505 and
I realize it need to repeat integration tests in multiple places.
Let's say instead we pivot the above matrix as follows:
```
| saving | errorwrapping | logging | ...
dialer | | | | ...
quicdialer | | | | ...
logging | | | | ...
mocking/sys | | | | ...
...
```
In this way, now every row contains everything related to a specific
action to perform. We can now share code without relying on extra
support packages. What's more, we can write tests and, judding from
the way in which things are made, it seems we only need integration
testing in `errorwrapping` because it's where data quality matters
whereas, in all other cases, unit testing is fine.
I am going, therefore, to proceed with these changes and "pivot"
`netx`. Hopefully, it won't be too painful.
2021-06-23 15:53:12 +02:00
|
|
|
}
|
|
|
|
|
2022-01-03 13:53:23 +01:00
|
|
|
var _ model.Dialer = &dialerLogger{}
|
refactor: start pivoting netx (#396)
What do I mean by pivoting? Netx is currently organized by row:
```
| dialer | quicdialer | resolver | ...
saving | | | | ...
errorwrapping | | | | ...
logging | | | | ...
mocking/sys | | | | ...
```
Every row needs to implement saving, errorwrapping, logging, mocking (or
adapting to the system or to some underlying library).
This causes cross package dependencies and, in turn, complexity. For
example, we need the `trace` package for supporting saving.
And `dialer`, `quickdialer`, et al. need to depend on such a package.
The same goes for errorwrapping.
This arrangement further complicates testing. For example, I am
currently working on https://github.com/ooni/probe/issues/1505 and
I realize it need to repeat integration tests in multiple places.
Let's say instead we pivot the above matrix as follows:
```
| saving | errorwrapping | logging | ...
dialer | | | | ...
quicdialer | | | | ...
logging | | | | ...
mocking/sys | | | | ...
...
```
In this way, now every row contains everything related to a specific
action to perform. We can now share code without relying on extra
support packages. What's more, we can write tests and, judding from
the way in which things are made, it seems we only need integration
testing in `errorwrapping` because it's where data quality matters
whereas, in all other cases, unit testing is fine.
I am going, therefore, to proceed with these changes and "pivot"
`netx`. Hopefully, it won't be too painful.
2021-06-23 15:53:12 +02:00
|
|
|
|
2021-09-05 14:49:38 +02:00
|
|
|
func (d *dialerLogger) DialContext(ctx context.Context, network, address string) (net.Conn, error) {
|
2022-01-03 13:53:23 +01:00
|
|
|
d.DebugLogger.Debugf("dial%s %s/%s...", d.operationSuffix, address, network)
|
refactor: start pivoting netx (#396)
What do I mean by pivoting? Netx is currently organized by row:
```
| dialer | quicdialer | resolver | ...
saving | | | | ...
errorwrapping | | | | ...
logging | | | | ...
mocking/sys | | | | ...
```
Every row needs to implement saving, errorwrapping, logging, mocking (or
adapting to the system or to some underlying library).
This causes cross package dependencies and, in turn, complexity. For
example, we need the `trace` package for supporting saving.
And `dialer`, `quickdialer`, et al. need to depend on such a package.
The same goes for errorwrapping.
This arrangement further complicates testing. For example, I am
currently working on https://github.com/ooni/probe/issues/1505 and
I realize it need to repeat integration tests in multiple places.
Let's say instead we pivot the above matrix as follows:
```
| saving | errorwrapping | logging | ...
dialer | | | | ...
quicdialer | | | | ...
logging | | | | ...
mocking/sys | | | | ...
...
```
In this way, now every row contains everything related to a specific
action to perform. We can now share code without relying on extra
support packages. What's more, we can write tests and, judding from
the way in which things are made, it seems we only need integration
testing in `errorwrapping` because it's where data quality matters
whereas, in all other cases, unit testing is fine.
I am going, therefore, to proceed with these changes and "pivot"
`netx`. Hopefully, it won't be too painful.
2021-06-23 15:53:12 +02:00
|
|
|
start := time.Now()
|
|
|
|
conn, err := d.Dialer.DialContext(ctx, network, address)
|
2021-06-23 17:00:44 +02:00
|
|
|
elapsed := time.Since(start)
|
|
|
|
if err != nil {
|
2022-01-03 13:53:23 +01:00
|
|
|
d.DebugLogger.Debugf("dial%s %s/%s... %s in %s", d.operationSuffix,
|
2021-09-06 21:34:14 +02:00
|
|
|
address, network, err, elapsed)
|
2021-06-23 17:00:44 +02:00
|
|
|
return nil, err
|
|
|
|
}
|
2022-01-03 13:53:23 +01:00
|
|
|
d.DebugLogger.Debugf("dial%s %s/%s... ok in %s", d.operationSuffix,
|
2021-09-06 21:34:14 +02:00
|
|
|
address, network, elapsed)
|
2021-06-23 17:00:44 +02:00
|
|
|
return conn, nil
|
refactor: start pivoting netx (#396)
What do I mean by pivoting? Netx is currently organized by row:
```
| dialer | quicdialer | resolver | ...
saving | | | | ...
errorwrapping | | | | ...
logging | | | | ...
mocking/sys | | | | ...
```
Every row needs to implement saving, errorwrapping, logging, mocking (or
adapting to the system or to some underlying library).
This causes cross package dependencies and, in turn, complexity. For
example, we need the `trace` package for supporting saving.
And `dialer`, `quickdialer`, et al. need to depend on such a package.
The same goes for errorwrapping.
This arrangement further complicates testing. For example, I am
currently working on https://github.com/ooni/probe/issues/1505 and
I realize it need to repeat integration tests in multiple places.
Let's say instead we pivot the above matrix as follows:
```
| saving | errorwrapping | logging | ...
dialer | | | | ...
quicdialer | | | | ...
logging | | | | ...
mocking/sys | | | | ...
...
```
In this way, now every row contains everything related to a specific
action to perform. We can now share code without relying on extra
support packages. What's more, we can write tests and, judding from
the way in which things are made, it seems we only need integration
testing in `errorwrapping` because it's where data quality matters
whereas, in all other cases, unit testing is fine.
I am going, therefore, to proceed with these changes and "pivot"
`netx`. Hopefully, it won't be too painful.
2021-06-23 15:53:12 +02:00
|
|
|
}
|
2021-09-05 19:55:28 +02:00
|
|
|
|
|
|
|
func (d *dialerLogger) CloseIdleConnections() {
|
|
|
|
d.Dialer.CloseIdleConnections()
|
|
|
|
}
|
2021-09-06 14:12:30 +02:00
|
|
|
|
2021-09-29 20:21:25 +02:00
|
|
|
// ErrNoConnReuse is the type of error returned when you create a
|
|
|
|
// "single use" dialer or a "single use" TLS dialer and you dial
|
|
|
|
// more than once, which is not supported by such a dialer.
|
2021-09-06 14:12:30 +02:00
|
|
|
var ErrNoConnReuse = errors.New("cannot reuse connection")
|
|
|
|
|
2021-09-29 20:21:25 +02:00
|
|
|
// NewSingleUseDialer returns a "single use" dialer. The first
|
|
|
|
// dial will succed and return conn regardless of the network
|
|
|
|
// and address arguments passed to DialContext. Any subsequent
|
|
|
|
// dial returns ErrNoConnReuse.
|
2022-01-03 13:53:23 +01:00
|
|
|
func NewSingleUseDialer(conn net.Conn) model.Dialer {
|
2021-09-06 14:12:30 +02:00
|
|
|
return &dialerSingleUse{conn: conn}
|
|
|
|
}
|
|
|
|
|
2021-09-06 22:14:49 +02:00
|
|
|
// dialerSingleUse is the Dialer returned by NewSingleDialer.
|
2021-09-06 14:12:30 +02:00
|
|
|
type dialerSingleUse struct {
|
2022-05-15 19:25:27 +02:00
|
|
|
mu sync.Mutex
|
2021-09-06 14:12:30 +02:00
|
|
|
conn net.Conn
|
|
|
|
}
|
|
|
|
|
2022-01-03 13:53:23 +01:00
|
|
|
var _ model.Dialer = &dialerSingleUse{}
|
2021-09-06 14:12:30 +02:00
|
|
|
|
|
|
|
func (s *dialerSingleUse) DialContext(ctx context.Context, network string, addr string) (net.Conn, error) {
|
2022-05-15 19:25:27 +02:00
|
|
|
defer s.mu.Unlock()
|
|
|
|
s.mu.Lock()
|
2021-09-06 14:12:30 +02:00
|
|
|
if s.conn == nil {
|
|
|
|
return nil, ErrNoConnReuse
|
|
|
|
}
|
|
|
|
var conn net.Conn
|
|
|
|
conn, s.conn = s.conn, nil
|
|
|
|
return conn, nil
|
|
|
|
}
|
|
|
|
|
|
|
|
func (s *dialerSingleUse) CloseIdleConnections() {
|
2021-09-08 21:19:51 +02:00
|
|
|
// nothing to do
|
2021-09-06 14:12:30 +02:00
|
|
|
}
|
2021-09-07 19:56:42 +02:00
|
|
|
|
|
|
|
// dialerErrWrapper is a dialer that performs error wrapping. The connection
|
|
|
|
// returned by the DialContext function will also perform error wrapping.
|
|
|
|
type dialerErrWrapper struct {
|
2022-05-15 19:25:27 +02:00
|
|
|
Dialer model.Dialer
|
2021-09-07 19:56:42 +02:00
|
|
|
}
|
|
|
|
|
2022-01-03 13:53:23 +01:00
|
|
|
var _ model.Dialer = &dialerErrWrapper{}
|
2021-09-07 19:56:42 +02:00
|
|
|
|
|
|
|
func (d *dialerErrWrapper) DialContext(ctx context.Context, network, address string) (net.Conn, error) {
|
|
|
|
conn, err := d.Dialer.DialContext(ctx, network, address)
|
|
|
|
if err != nil {
|
2022-05-15 19:25:27 +02:00
|
|
|
return nil, newErrWrapper(classifyGenericError, ConnectOperation, err)
|
2021-09-07 19:56:42 +02:00
|
|
|
}
|
|
|
|
return &dialerErrWrapperConn{Conn: conn}, nil
|
|
|
|
}
|
|
|
|
|
2022-05-15 19:25:27 +02:00
|
|
|
func (d *dialerErrWrapper) CloseIdleConnections() {
|
|
|
|
d.Dialer.CloseIdleConnections()
|
|
|
|
}
|
|
|
|
|
2021-09-07 19:56:42 +02:00
|
|
|
// dialerErrWrapperConn is a net.Conn that performs error wrapping.
|
|
|
|
type dialerErrWrapperConn struct {
|
|
|
|
net.Conn
|
|
|
|
}
|
|
|
|
|
|
|
|
var _ net.Conn = &dialerErrWrapperConn{}
|
|
|
|
|
|
|
|
func (c *dialerErrWrapperConn) Read(b []byte) (int, error) {
|
|
|
|
count, err := c.Conn.Read(b)
|
|
|
|
if err != nil {
|
2022-05-15 19:25:27 +02:00
|
|
|
return 0, newErrWrapper(classifyGenericError, ReadOperation, err)
|
2021-09-07 19:56:42 +02:00
|
|
|
}
|
|
|
|
return count, nil
|
|
|
|
}
|
|
|
|
|
|
|
|
func (c *dialerErrWrapperConn) Write(b []byte) (int, error) {
|
|
|
|
count, err := c.Conn.Write(b)
|
|
|
|
if err != nil {
|
2022-05-15 19:25:27 +02:00
|
|
|
return 0, newErrWrapper(classifyGenericError, WriteOperation, err)
|
2021-09-07 19:56:42 +02:00
|
|
|
}
|
|
|
|
return count, nil
|
|
|
|
}
|
|
|
|
|
|
|
|
func (c *dialerErrWrapperConn) Close() error {
|
|
|
|
err := c.Conn.Close()
|
|
|
|
if err != nil {
|
2022-05-15 19:25:27 +02:00
|
|
|
return newErrWrapper(classifyGenericError, CloseOperation, err)
|
2021-09-07 19:56:42 +02:00
|
|
|
}
|
|
|
|
return nil
|
|
|
|
}
|
2021-09-08 14:46:17 +02:00
|
|
|
|
2021-09-29 20:21:25 +02:00
|
|
|
// ErrNoDialer is the type of error returned by "null" dialers
|
|
|
|
// when you attempt to dial with them.
|
2021-09-08 14:46:17 +02:00
|
|
|
var ErrNoDialer = errors.New("no configured dialer")
|
|
|
|
|
2021-09-29 20:21:25 +02:00
|
|
|
// NewNullDialer returns a dialer that always fails with ErrNoDialer.
|
2022-01-03 13:53:23 +01:00
|
|
|
func NewNullDialer() model.Dialer {
|
2021-09-08 14:46:17 +02:00
|
|
|
return &nullDialer{}
|
|
|
|
}
|
|
|
|
|
|
|
|
type nullDialer struct{}
|
|
|
|
|
2022-01-03 13:53:23 +01:00
|
|
|
var _ model.Dialer = &nullDialer{}
|
2021-09-08 14:46:17 +02:00
|
|
|
|
|
|
|
func (*nullDialer) DialContext(ctx context.Context, network, address string) (net.Conn, error) {
|
|
|
|
return nil, ErrNoDialer
|
|
|
|
}
|
|
|
|
|
|
|
|
func (*nullDialer) CloseIdleConnections() {
|
|
|
|
// nothing to do
|
|
|
|
}
|