refactor: start pivoting netx (#396)
What do I mean by pivoting? Netx is currently organized by row:
```
| dialer | quicdialer | resolver | ...
saving | | | | ...
errorwrapping | | | | ...
logging | | | | ...
mocking/sys | | | | ...
```
Every row needs to implement saving, errorwrapping, logging, mocking (or
adapting to the system or to some underlying library).
This causes cross package dependencies and, in turn, complexity. For
example, we need the `trace` package for supporting saving.
And `dialer`, `quickdialer`, et al. need to depend on such a package.
The same goes for errorwrapping.
This arrangement further complicates testing. For example, I am
currently working on https://github.com/ooni/probe/issues/1505 and
I realize it need to repeat integration tests in multiple places.
Let's say instead we pivot the above matrix as follows:
```
| saving | errorwrapping | logging | ...
dialer | | | | ...
quicdialer | | | | ...
logging | | | | ...
mocking/sys | | | | ...
...
```
In this way, now every row contains everything related to a specific
action to perform. We can now share code without relying on extra
support packages. What's more, we can write tests and, judding from
the way in which things are made, it seems we only need integration
testing in `errorwrapping` because it's where data quality matters
whereas, in all other cases, unit testing is fine.
I am going, therefore, to proceed with these changes and "pivot"
`netx`. Hopefully, it won't be too painful.
2021-06-23 15:53:12 +02:00
|
|
|
package netxlite
|
|
|
|
|
|
|
|
import (
|
|
|
|
"context"
|
2021-09-06 14:12:30 +02:00
|
|
|
"errors"
|
refactor: start pivoting netx (#396)
What do I mean by pivoting? Netx is currently organized by row:
```
| dialer | quicdialer | resolver | ...
saving | | | | ...
errorwrapping | | | | ...
logging | | | | ...
mocking/sys | | | | ...
```
Every row needs to implement saving, errorwrapping, logging, mocking (or
adapting to the system or to some underlying library).
This causes cross package dependencies and, in turn, complexity. For
example, we need the `trace` package for supporting saving.
And `dialer`, `quickdialer`, et al. need to depend on such a package.
The same goes for errorwrapping.
This arrangement further complicates testing. For example, I am
currently working on https://github.com/ooni/probe/issues/1505 and
I realize it need to repeat integration tests in multiple places.
Let's say instead we pivot the above matrix as follows:
```
| saving | errorwrapping | logging | ...
dialer | | | | ...
quicdialer | | | | ...
logging | | | | ...
mocking/sys | | | | ...
...
```
In this way, now every row contains everything related to a specific
action to perform. We can now share code without relying on extra
support packages. What's more, we can write tests and, judding from
the way in which things are made, it seems we only need integration
testing in `errorwrapping` because it's where data quality matters
whereas, in all other cases, unit testing is fine.
I am going, therefore, to proceed with these changes and "pivot"
`netx`. Hopefully, it won't be too painful.
2021-06-23 15:53:12 +02:00
|
|
|
"net"
|
2021-09-06 14:12:30 +02:00
|
|
|
"sync"
|
refactor: start pivoting netx (#396)
What do I mean by pivoting? Netx is currently organized by row:
```
| dialer | quicdialer | resolver | ...
saving | | | | ...
errorwrapping | | | | ...
logging | | | | ...
mocking/sys | | | | ...
```
Every row needs to implement saving, errorwrapping, logging, mocking (or
adapting to the system or to some underlying library).
This causes cross package dependencies and, in turn, complexity. For
example, we need the `trace` package for supporting saving.
And `dialer`, `quickdialer`, et al. need to depend on such a package.
The same goes for errorwrapping.
This arrangement further complicates testing. For example, I am
currently working on https://github.com/ooni/probe/issues/1505 and
I realize it need to repeat integration tests in multiple places.
Let's say instead we pivot the above matrix as follows:
```
| saving | errorwrapping | logging | ...
dialer | | | | ...
quicdialer | | | | ...
logging | | | | ...
mocking/sys | | | | ...
...
```
In this way, now every row contains everything related to a specific
action to perform. We can now share code without relying on extra
support packages. What's more, we can write tests and, judding from
the way in which things are made, it seems we only need integration
testing in `errorwrapping` because it's where data quality matters
whereas, in all other cases, unit testing is fine.
I am going, therefore, to proceed with these changes and "pivot"
`netx`. Hopefully, it won't be too painful.
2021-06-23 15:53:12 +02:00
|
|
|
"time"
|
2021-09-07 19:56:42 +02:00
|
|
|
|
|
|
|
"github.com/ooni/probe-cli/v3/internal/netxlite/errorsx"
|
refactor: start pivoting netx (#396)
What do I mean by pivoting? Netx is currently organized by row:
```
| dialer | quicdialer | resolver | ...
saving | | | | ...
errorwrapping | | | | ...
logging | | | | ...
mocking/sys | | | | ...
```
Every row needs to implement saving, errorwrapping, logging, mocking (or
adapting to the system or to some underlying library).
This causes cross package dependencies and, in turn, complexity. For
example, we need the `trace` package for supporting saving.
And `dialer`, `quickdialer`, et al. need to depend on such a package.
The same goes for errorwrapping.
This arrangement further complicates testing. For example, I am
currently working on https://github.com/ooni/probe/issues/1505 and
I realize it need to repeat integration tests in multiple places.
Let's say instead we pivot the above matrix as follows:
```
| saving | errorwrapping | logging | ...
dialer | | | | ...
quicdialer | | | | ...
logging | | | | ...
mocking/sys | | | | ...
...
```
In this way, now every row contains everything related to a specific
action to perform. We can now share code without relying on extra
support packages. What's more, we can write tests and, judding from
the way in which things are made, it seems we only need integration
testing in `errorwrapping` because it's where data quality matters
whereas, in all other cases, unit testing is fine.
I am going, therefore, to proceed with these changes and "pivot"
`netx`. Hopefully, it won't be too painful.
2021-06-23 15:53:12 +02:00
|
|
|
)
|
|
|
|
|
|
|
|
// Dialer establishes network connections.
|
|
|
|
type Dialer interface {
|
|
|
|
// DialContext behaves like net.Dialer.DialContext.
|
|
|
|
DialContext(ctx context.Context, network, address string) (net.Conn, error)
|
2021-09-05 19:55:28 +02:00
|
|
|
|
|
|
|
// CloseIdleConnections closes idle connections, if any.
|
|
|
|
CloseIdleConnections()
|
refactor: start pivoting netx (#396)
What do I mean by pivoting? Netx is currently organized by row:
```
| dialer | quicdialer | resolver | ...
saving | | | | ...
errorwrapping | | | | ...
logging | | | | ...
mocking/sys | | | | ...
```
Every row needs to implement saving, errorwrapping, logging, mocking (or
adapting to the system or to some underlying library).
This causes cross package dependencies and, in turn, complexity. For
example, we need the `trace` package for supporting saving.
And `dialer`, `quickdialer`, et al. need to depend on such a package.
The same goes for errorwrapping.
This arrangement further complicates testing. For example, I am
currently working on https://github.com/ooni/probe/issues/1505 and
I realize it need to repeat integration tests in multiple places.
Let's say instead we pivot the above matrix as follows:
```
| saving | errorwrapping | logging | ...
dialer | | | | ...
quicdialer | | | | ...
logging | | | | ...
mocking/sys | | | | ...
...
```
In this way, now every row contains everything related to a specific
action to perform. We can now share code without relying on extra
support packages. What's more, we can write tests and, judding from
the way in which things are made, it seems we only need integration
testing in `errorwrapping` because it's where data quality matters
whereas, in all other cases, unit testing is fine.
I am going, therefore, to proceed with these changes and "pivot"
`netx`. Hopefully, it won't be too painful.
2021-06-23 15:53:12 +02:00
|
|
|
}
|
|
|
|
|
2021-09-08 00:00:53 +02:00
|
|
|
// NewDialerWithResolver creates a new Dialer. The returned Dialer
|
|
|
|
// has the following properties:
|
|
|
|
//
|
2021-09-08 21:19:51 +02:00
|
|
|
// 1. logs events using the given logger;
|
2021-09-08 00:00:53 +02:00
|
|
|
//
|
2021-09-08 21:19:51 +02:00
|
|
|
// 2. resolves domain names using the givern resolver;
|
2021-09-08 00:00:53 +02:00
|
|
|
//
|
2021-09-08 21:19:51 +02:00
|
|
|
// 3. when using a resolver, each available enpoint is tried
|
|
|
|
// sequentially. On error, the code will return what it believes
|
|
|
|
// to be the most representative error in the pack. Most often,
|
|
|
|
// such an error is the first one that occurred. Choosing the
|
|
|
|
// error to return using this logic is a QUIRK that we owe
|
|
|
|
// to the original implementation of netx. We cannot change
|
|
|
|
// this behavior until all the legacy code that relies on
|
|
|
|
// it has been migrated to more sane patterns.
|
2021-09-08 00:00:53 +02:00
|
|
|
//
|
2021-09-08 21:19:51 +02:00
|
|
|
// Removing this quirk from the codebase is documented as
|
|
|
|
// TODO(https://github.com/ooni/probe/issues/1779).
|
|
|
|
//
|
|
|
|
// 4. wraps errors;
|
|
|
|
//
|
|
|
|
// 5. has a configured connect timeout.
|
2021-09-05 20:41:46 +02:00
|
|
|
func NewDialerWithResolver(logger Logger, resolver Resolver) Dialer {
|
|
|
|
return &dialerLogger{
|
|
|
|
Dialer: &dialerResolver{
|
|
|
|
Dialer: &dialerLogger{
|
2021-09-07 19:56:42 +02:00
|
|
|
Dialer: &dialerErrWrapper{
|
|
|
|
Dialer: &dialerSystem{},
|
|
|
|
},
|
2021-09-06 21:34:14 +02:00
|
|
|
Logger: logger,
|
|
|
|
operationSuffix: "_address",
|
2021-09-05 20:41:46 +02:00
|
|
|
},
|
|
|
|
Resolver: resolver,
|
|
|
|
},
|
|
|
|
Logger: logger,
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2021-09-08 00:00:53 +02:00
|
|
|
// NewDialerWithoutResolver is like NewDialerWithResolver except that
|
|
|
|
// it will fail with ErrNoResolver if passed a domain name.
|
2021-09-05 20:41:46 +02:00
|
|
|
func NewDialerWithoutResolver(logger Logger) Dialer {
|
|
|
|
return NewDialerWithResolver(logger, &nullResolver{})
|
|
|
|
}
|
|
|
|
|
2021-09-08 21:19:51 +02:00
|
|
|
// dialerSystem uses system facilities to perform domain name
|
|
|
|
// resolution and guarantees we have a dialer timeout.
|
2021-09-08 00:00:53 +02:00
|
|
|
type dialerSystem struct {
|
|
|
|
// timeout is the OPTIONAL timeout used for testing.
|
|
|
|
timeout time.Duration
|
refactor: start pivoting netx (#396)
What do I mean by pivoting? Netx is currently organized by row:
```
| dialer | quicdialer | resolver | ...
saving | | | | ...
errorwrapping | | | | ...
logging | | | | ...
mocking/sys | | | | ...
```
Every row needs to implement saving, errorwrapping, logging, mocking (or
adapting to the system or to some underlying library).
This causes cross package dependencies and, in turn, complexity. For
example, we need the `trace` package for supporting saving.
And `dialer`, `quickdialer`, et al. need to depend on such a package.
The same goes for errorwrapping.
This arrangement further complicates testing. For example, I am
currently working on https://github.com/ooni/probe/issues/1505 and
I realize it need to repeat integration tests in multiple places.
Let's say instead we pivot the above matrix as follows:
```
| saving | errorwrapping | logging | ...
dialer | | | | ...
quicdialer | | | | ...
logging | | | | ...
mocking/sys | | | | ...
...
```
In this way, now every row contains everything related to a specific
action to perform. We can now share code without relying on extra
support packages. What's more, we can write tests and, judding from
the way in which things are made, it seems we only need integration
testing in `errorwrapping` because it's where data quality matters
whereas, in all other cases, unit testing is fine.
I am going, therefore, to proceed with these changes and "pivot"
`netx`. Hopefully, it won't be too painful.
2021-06-23 15:53:12 +02:00
|
|
|
}
|
|
|
|
|
2021-09-08 21:19:51 +02:00
|
|
|
var _ Dialer = &dialerSystem{}
|
|
|
|
|
|
|
|
const dialerDefaultTimeout = 15 * time.Second
|
|
|
|
|
2021-09-08 00:00:53 +02:00
|
|
|
func (d *dialerSystem) newUnderlyingDialer() *net.Dialer {
|
|
|
|
t := d.timeout
|
|
|
|
if t <= 0 {
|
2021-09-08 21:19:51 +02:00
|
|
|
t = dialerDefaultTimeout
|
2021-09-08 00:00:53 +02:00
|
|
|
}
|
|
|
|
return &net.Dialer{Timeout: t}
|
|
|
|
}
|
2021-09-05 19:55:28 +02:00
|
|
|
|
|
|
|
func (d *dialerSystem) DialContext(ctx context.Context, network, address string) (net.Conn, error) {
|
2021-09-08 00:00:53 +02:00
|
|
|
return d.newUnderlyingDialer().DialContext(ctx, network, address)
|
2021-09-05 19:55:28 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
func (d *dialerSystem) CloseIdleConnections() {
|
2021-09-08 21:19:51 +02:00
|
|
|
// nothing to do here
|
2021-09-05 19:55:28 +02:00
|
|
|
}
|
|
|
|
|
2021-09-08 21:19:51 +02:00
|
|
|
// dialerResolver combines dialing with domain name resolution.
|
2021-09-05 14:49:38 +02:00
|
|
|
type dialerResolver struct {
|
2021-09-08 21:19:51 +02:00
|
|
|
Dialer
|
|
|
|
Resolver
|
refactor: start pivoting netx (#396)
What do I mean by pivoting? Netx is currently organized by row:
```
| dialer | quicdialer | resolver | ...
saving | | | | ...
errorwrapping | | | | ...
logging | | | | ...
mocking/sys | | | | ...
```
Every row needs to implement saving, errorwrapping, logging, mocking (or
adapting to the system or to some underlying library).
This causes cross package dependencies and, in turn, complexity. For
example, we need the `trace` package for supporting saving.
And `dialer`, `quickdialer`, et al. need to depend on such a package.
The same goes for errorwrapping.
This arrangement further complicates testing. For example, I am
currently working on https://github.com/ooni/probe/issues/1505 and
I realize it need to repeat integration tests in multiple places.
Let's say instead we pivot the above matrix as follows:
```
| saving | errorwrapping | logging | ...
dialer | | | | ...
quicdialer | | | | ...
logging | | | | ...
mocking/sys | | | | ...
...
```
In this way, now every row contains everything related to a specific
action to perform. We can now share code without relying on extra
support packages. What's more, we can write tests and, judding from
the way in which things are made, it seems we only need integration
testing in `errorwrapping` because it's where data quality matters
whereas, in all other cases, unit testing is fine.
I am going, therefore, to proceed with these changes and "pivot"
`netx`. Hopefully, it won't be too painful.
2021-06-23 15:53:12 +02:00
|
|
|
}
|
|
|
|
|
2021-09-05 14:49:38 +02:00
|
|
|
var _ Dialer = &dialerResolver{}
|
refactor: start pivoting netx (#396)
What do I mean by pivoting? Netx is currently organized by row:
```
| dialer | quicdialer | resolver | ...
saving | | | | ...
errorwrapping | | | | ...
logging | | | | ...
mocking/sys | | | | ...
```
Every row needs to implement saving, errorwrapping, logging, mocking (or
adapting to the system or to some underlying library).
This causes cross package dependencies and, in turn, complexity. For
example, we need the `trace` package for supporting saving.
And `dialer`, `quickdialer`, et al. need to depend on such a package.
The same goes for errorwrapping.
This arrangement further complicates testing. For example, I am
currently working on https://github.com/ooni/probe/issues/1505 and
I realize it need to repeat integration tests in multiple places.
Let's say instead we pivot the above matrix as follows:
```
| saving | errorwrapping | logging | ...
dialer | | | | ...
quicdialer | | | | ...
logging | | | | ...
mocking/sys | | | | ...
...
```
In this way, now every row contains everything related to a specific
action to perform. We can now share code without relying on extra
support packages. What's more, we can write tests and, judding from
the way in which things are made, it seems we only need integration
testing in `errorwrapping` because it's where data quality matters
whereas, in all other cases, unit testing is fine.
I am going, therefore, to proceed with these changes and "pivot"
`netx`. Hopefully, it won't be too painful.
2021-06-23 15:53:12 +02:00
|
|
|
|
2021-09-05 14:49:38 +02:00
|
|
|
func (d *dialerResolver) DialContext(ctx context.Context, network, address string) (net.Conn, error) {
|
2021-09-08 21:19:51 +02:00
|
|
|
// QUIRK: this routine and the related routines in quirks.go cannot
|
|
|
|
// be changed easily until we use events tracing to measure.
|
|
|
|
//
|
|
|
|
// Reference issue: TODO(https://github.com/ooni/probe/issues/1779).
|
refactor: start pivoting netx (#396)
What do I mean by pivoting? Netx is currently organized by row:
```
| dialer | quicdialer | resolver | ...
saving | | | | ...
errorwrapping | | | | ...
logging | | | | ...
mocking/sys | | | | ...
```
Every row needs to implement saving, errorwrapping, logging, mocking (or
adapting to the system or to some underlying library).
This causes cross package dependencies and, in turn, complexity. For
example, we need the `trace` package for supporting saving.
And `dialer`, `quickdialer`, et al. need to depend on such a package.
The same goes for errorwrapping.
This arrangement further complicates testing. For example, I am
currently working on https://github.com/ooni/probe/issues/1505 and
I realize it need to repeat integration tests in multiple places.
Let's say instead we pivot the above matrix as follows:
```
| saving | errorwrapping | logging | ...
dialer | | | | ...
quicdialer | | | | ...
logging | | | | ...
mocking/sys | | | | ...
...
```
In this way, now every row contains everything related to a specific
action to perform. We can now share code without relying on extra
support packages. What's more, we can write tests and, judding from
the way in which things are made, it seems we only need integration
testing in `errorwrapping` because it's where data quality matters
whereas, in all other cases, unit testing is fine.
I am going, therefore, to proceed with these changes and "pivot"
`netx`. Hopefully, it won't be too painful.
2021-06-23 15:53:12 +02:00
|
|
|
onlyhost, onlyport, err := net.SplitHostPort(address)
|
|
|
|
if err != nil {
|
|
|
|
return nil, err
|
|
|
|
}
|
2021-06-23 17:00:44 +02:00
|
|
|
addrs, err := d.lookupHost(ctx, onlyhost)
|
refactor: start pivoting netx (#396)
What do I mean by pivoting? Netx is currently organized by row:
```
| dialer | quicdialer | resolver | ...
saving | | | | ...
errorwrapping | | | | ...
logging | | | | ...
mocking/sys | | | | ...
```
Every row needs to implement saving, errorwrapping, logging, mocking (or
adapting to the system or to some underlying library).
This causes cross package dependencies and, in turn, complexity. For
example, we need the `trace` package for supporting saving.
And `dialer`, `quickdialer`, et al. need to depend on such a package.
The same goes for errorwrapping.
This arrangement further complicates testing. For example, I am
currently working on https://github.com/ooni/probe/issues/1505 and
I realize it need to repeat integration tests in multiple places.
Let's say instead we pivot the above matrix as follows:
```
| saving | errorwrapping | logging | ...
dialer | | | | ...
quicdialer | | | | ...
logging | | | | ...
mocking/sys | | | | ...
...
```
In this way, now every row contains everything related to a specific
action to perform. We can now share code without relying on extra
support packages. What's more, we can write tests and, judding from
the way in which things are made, it seems we only need integration
testing in `errorwrapping` because it's where data quality matters
whereas, in all other cases, unit testing is fine.
I am going, therefore, to proceed with these changes and "pivot"
`netx`. Hopefully, it won't be too painful.
2021-06-23 15:53:12 +02:00
|
|
|
if err != nil {
|
|
|
|
return nil, err
|
|
|
|
}
|
2021-09-06 20:17:45 +02:00
|
|
|
addrs = quirkSortIPAddrs(addrs)
|
refactor: start pivoting netx (#396)
What do I mean by pivoting? Netx is currently organized by row:
```
| dialer | quicdialer | resolver | ...
saving | | | | ...
errorwrapping | | | | ...
logging | | | | ...
mocking/sys | | | | ...
```
Every row needs to implement saving, errorwrapping, logging, mocking (or
adapting to the system or to some underlying library).
This causes cross package dependencies and, in turn, complexity. For
example, we need the `trace` package for supporting saving.
And `dialer`, `quickdialer`, et al. need to depend on such a package.
The same goes for errorwrapping.
This arrangement further complicates testing. For example, I am
currently working on https://github.com/ooni/probe/issues/1505 and
I realize it need to repeat integration tests in multiple places.
Let's say instead we pivot the above matrix as follows:
```
| saving | errorwrapping | logging | ...
dialer | | | | ...
quicdialer | | | | ...
logging | | | | ...
mocking/sys | | | | ...
...
```
In this way, now every row contains everything related to a specific
action to perform. We can now share code without relying on extra
support packages. What's more, we can write tests and, judding from
the way in which things are made, it seems we only need integration
testing in `errorwrapping` because it's where data quality matters
whereas, in all other cases, unit testing is fine.
I am going, therefore, to proceed with these changes and "pivot"
`netx`. Hopefully, it won't be too painful.
2021-06-23 15:53:12 +02:00
|
|
|
var errorslist []error
|
|
|
|
for _, addr := range addrs {
|
|
|
|
target := net.JoinHostPort(addr, onlyport)
|
|
|
|
conn, err := d.Dialer.DialContext(ctx, network, target)
|
|
|
|
if err == nil {
|
|
|
|
return conn, nil
|
|
|
|
}
|
|
|
|
errorslist = append(errorslist, err)
|
|
|
|
}
|
2021-09-06 20:17:45 +02:00
|
|
|
return nil, quirkReduceErrors(errorslist)
|
refactor: start pivoting netx (#396)
What do I mean by pivoting? Netx is currently organized by row:
```
| dialer | quicdialer | resolver | ...
saving | | | | ...
errorwrapping | | | | ...
logging | | | | ...
mocking/sys | | | | ...
```
Every row needs to implement saving, errorwrapping, logging, mocking (or
adapting to the system or to some underlying library).
This causes cross package dependencies and, in turn, complexity. For
example, we need the `trace` package for supporting saving.
And `dialer`, `quickdialer`, et al. need to depend on such a package.
The same goes for errorwrapping.
This arrangement further complicates testing. For example, I am
currently working on https://github.com/ooni/probe/issues/1505 and
I realize it need to repeat integration tests in multiple places.
Let's say instead we pivot the above matrix as follows:
```
| saving | errorwrapping | logging | ...
dialer | | | | ...
quicdialer | | | | ...
logging | | | | ...
mocking/sys | | | | ...
...
```
In this way, now every row contains everything related to a specific
action to perform. We can now share code without relying on extra
support packages. What's more, we can write tests and, judding from
the way in which things are made, it seems we only need integration
testing in `errorwrapping` because it's where data quality matters
whereas, in all other cases, unit testing is fine.
I am going, therefore, to proceed with these changes and "pivot"
`netx`. Hopefully, it won't be too painful.
2021-06-23 15:53:12 +02:00
|
|
|
}
|
|
|
|
|
2021-09-08 21:19:51 +02:00
|
|
|
// lookupHost ensures we correctly handle IP addresses.
|
2021-09-05 14:49:38 +02:00
|
|
|
func (d *dialerResolver) lookupHost(ctx context.Context, hostname string) ([]string, error) {
|
refactor: start pivoting netx (#396)
What do I mean by pivoting? Netx is currently organized by row:
```
| dialer | quicdialer | resolver | ...
saving | | | | ...
errorwrapping | | | | ...
logging | | | | ...
mocking/sys | | | | ...
```
Every row needs to implement saving, errorwrapping, logging, mocking (or
adapting to the system or to some underlying library).
This causes cross package dependencies and, in turn, complexity. For
example, we need the `trace` package for supporting saving.
And `dialer`, `quickdialer`, et al. need to depend on such a package.
The same goes for errorwrapping.
This arrangement further complicates testing. For example, I am
currently working on https://github.com/ooni/probe/issues/1505 and
I realize it need to repeat integration tests in multiple places.
Let's say instead we pivot the above matrix as follows:
```
| saving | errorwrapping | logging | ...
dialer | | | | ...
quicdialer | | | | ...
logging | | | | ...
mocking/sys | | | | ...
...
```
In this way, now every row contains everything related to a specific
action to perform. We can now share code without relying on extra
support packages. What's more, we can write tests and, judding from
the way in which things are made, it seems we only need integration
testing in `errorwrapping` because it's where data quality matters
whereas, in all other cases, unit testing is fine.
I am going, therefore, to proceed with these changes and "pivot"
`netx`. Hopefully, it won't be too painful.
2021-06-23 15:53:12 +02:00
|
|
|
if net.ParseIP(hostname) != nil {
|
|
|
|
return []string{hostname}, nil
|
|
|
|
}
|
|
|
|
return d.Resolver.LookupHost(ctx, hostname)
|
|
|
|
}
|
|
|
|
|
2021-09-05 19:55:28 +02:00
|
|
|
func (d *dialerResolver) CloseIdleConnections() {
|
|
|
|
d.Dialer.CloseIdleConnections()
|
|
|
|
d.Resolver.CloseIdleConnections()
|
|
|
|
}
|
|
|
|
|
2021-09-05 14:49:38 +02:00
|
|
|
// dialerLogger is a Dialer with logging.
|
|
|
|
type dialerLogger struct {
|
2021-06-23 17:00:44 +02:00
|
|
|
// Dialer is the underlying dialer.
|
2021-09-08 21:19:51 +02:00
|
|
|
Dialer
|
2021-06-23 17:00:44 +02:00
|
|
|
|
|
|
|
// Logger is the underlying logger.
|
2021-09-08 21:19:51 +02:00
|
|
|
Logger
|
2021-09-06 21:34:14 +02:00
|
|
|
|
|
|
|
// operationSuffix is appended to the operation name.
|
|
|
|
//
|
|
|
|
// We use this suffix to distinguish the output from dialing
|
|
|
|
// with the output from dialing an IP address when we are
|
|
|
|
// using a dialer without resolver, where otherwise both lines
|
|
|
|
// would read something like `dial 8.8.8.8:443...`
|
|
|
|
operationSuffix string
|
refactor: start pivoting netx (#396)
What do I mean by pivoting? Netx is currently organized by row:
```
| dialer | quicdialer | resolver | ...
saving | | | | ...
errorwrapping | | | | ...
logging | | | | ...
mocking/sys | | | | ...
```
Every row needs to implement saving, errorwrapping, logging, mocking (or
adapting to the system or to some underlying library).
This causes cross package dependencies and, in turn, complexity. For
example, we need the `trace` package for supporting saving.
And `dialer`, `quickdialer`, et al. need to depend on such a package.
The same goes for errorwrapping.
This arrangement further complicates testing. For example, I am
currently working on https://github.com/ooni/probe/issues/1505 and
I realize it need to repeat integration tests in multiple places.
Let's say instead we pivot the above matrix as follows:
```
| saving | errorwrapping | logging | ...
dialer | | | | ...
quicdialer | | | | ...
logging | | | | ...
mocking/sys | | | | ...
...
```
In this way, now every row contains everything related to a specific
action to perform. We can now share code without relying on extra
support packages. What's more, we can write tests and, judding from
the way in which things are made, it seems we only need integration
testing in `errorwrapping` because it's where data quality matters
whereas, in all other cases, unit testing is fine.
I am going, therefore, to proceed with these changes and "pivot"
`netx`. Hopefully, it won't be too painful.
2021-06-23 15:53:12 +02:00
|
|
|
}
|
|
|
|
|
2021-09-05 14:49:38 +02:00
|
|
|
var _ Dialer = &dialerLogger{}
|
refactor: start pivoting netx (#396)
What do I mean by pivoting? Netx is currently organized by row:
```
| dialer | quicdialer | resolver | ...
saving | | | | ...
errorwrapping | | | | ...
logging | | | | ...
mocking/sys | | | | ...
```
Every row needs to implement saving, errorwrapping, logging, mocking (or
adapting to the system or to some underlying library).
This causes cross package dependencies and, in turn, complexity. For
example, we need the `trace` package for supporting saving.
And `dialer`, `quickdialer`, et al. need to depend on such a package.
The same goes for errorwrapping.
This arrangement further complicates testing. For example, I am
currently working on https://github.com/ooni/probe/issues/1505 and
I realize it need to repeat integration tests in multiple places.
Let's say instead we pivot the above matrix as follows:
```
| saving | errorwrapping | logging | ...
dialer | | | | ...
quicdialer | | | | ...
logging | | | | ...
mocking/sys | | | | ...
...
```
In this way, now every row contains everything related to a specific
action to perform. We can now share code without relying on extra
support packages. What's more, we can write tests and, judding from
the way in which things are made, it seems we only need integration
testing in `errorwrapping` because it's where data quality matters
whereas, in all other cases, unit testing is fine.
I am going, therefore, to proceed with these changes and "pivot"
`netx`. Hopefully, it won't be too painful.
2021-06-23 15:53:12 +02:00
|
|
|
|
2021-09-05 14:49:38 +02:00
|
|
|
func (d *dialerLogger) DialContext(ctx context.Context, network, address string) (net.Conn, error) {
|
2021-09-06 21:34:14 +02:00
|
|
|
d.Logger.Debugf("dial%s %s/%s...", d.operationSuffix, address, network)
|
refactor: start pivoting netx (#396)
What do I mean by pivoting? Netx is currently organized by row:
```
| dialer | quicdialer | resolver | ...
saving | | | | ...
errorwrapping | | | | ...
logging | | | | ...
mocking/sys | | | | ...
```
Every row needs to implement saving, errorwrapping, logging, mocking (or
adapting to the system or to some underlying library).
This causes cross package dependencies and, in turn, complexity. For
example, we need the `trace` package for supporting saving.
And `dialer`, `quickdialer`, et al. need to depend on such a package.
The same goes for errorwrapping.
This arrangement further complicates testing. For example, I am
currently working on https://github.com/ooni/probe/issues/1505 and
I realize it need to repeat integration tests in multiple places.
Let's say instead we pivot the above matrix as follows:
```
| saving | errorwrapping | logging | ...
dialer | | | | ...
quicdialer | | | | ...
logging | | | | ...
mocking/sys | | | | ...
...
```
In this way, now every row contains everything related to a specific
action to perform. We can now share code without relying on extra
support packages. What's more, we can write tests and, judding from
the way in which things are made, it seems we only need integration
testing in `errorwrapping` because it's where data quality matters
whereas, in all other cases, unit testing is fine.
I am going, therefore, to proceed with these changes and "pivot"
`netx`. Hopefully, it won't be too painful.
2021-06-23 15:53:12 +02:00
|
|
|
start := time.Now()
|
|
|
|
conn, err := d.Dialer.DialContext(ctx, network, address)
|
2021-06-23 17:00:44 +02:00
|
|
|
elapsed := time.Since(start)
|
|
|
|
if err != nil {
|
2021-09-06 21:34:14 +02:00
|
|
|
d.Logger.Debugf("dial%s %s/%s... %s in %s", d.operationSuffix,
|
|
|
|
address, network, err, elapsed)
|
2021-06-23 17:00:44 +02:00
|
|
|
return nil, err
|
|
|
|
}
|
2021-09-06 21:34:14 +02:00
|
|
|
d.Logger.Debugf("dial%s %s/%s... ok in %s", d.operationSuffix,
|
|
|
|
address, network, elapsed)
|
2021-06-23 17:00:44 +02:00
|
|
|
return conn, nil
|
refactor: start pivoting netx (#396)
What do I mean by pivoting? Netx is currently organized by row:
```
| dialer | quicdialer | resolver | ...
saving | | | | ...
errorwrapping | | | | ...
logging | | | | ...
mocking/sys | | | | ...
```
Every row needs to implement saving, errorwrapping, logging, mocking (or
adapting to the system or to some underlying library).
This causes cross package dependencies and, in turn, complexity. For
example, we need the `trace` package for supporting saving.
And `dialer`, `quickdialer`, et al. need to depend on such a package.
The same goes for errorwrapping.
This arrangement further complicates testing. For example, I am
currently working on https://github.com/ooni/probe/issues/1505 and
I realize it need to repeat integration tests in multiple places.
Let's say instead we pivot the above matrix as follows:
```
| saving | errorwrapping | logging | ...
dialer | | | | ...
quicdialer | | | | ...
logging | | | | ...
mocking/sys | | | | ...
...
```
In this way, now every row contains everything related to a specific
action to perform. We can now share code without relying on extra
support packages. What's more, we can write tests and, judding from
the way in which things are made, it seems we only need integration
testing in `errorwrapping` because it's where data quality matters
whereas, in all other cases, unit testing is fine.
I am going, therefore, to proceed with these changes and "pivot"
`netx`. Hopefully, it won't be too painful.
2021-06-23 15:53:12 +02:00
|
|
|
}
|
2021-09-05 19:55:28 +02:00
|
|
|
|
|
|
|
func (d *dialerLogger) CloseIdleConnections() {
|
|
|
|
d.Dialer.CloseIdleConnections()
|
|
|
|
}
|
2021-09-06 14:12:30 +02:00
|
|
|
|
|
|
|
// ErrNoConnReuse indicates we cannot reuse the connection provided
|
|
|
|
// to a single use (possibly TLS) dialer.
|
|
|
|
var ErrNoConnReuse = errors.New("cannot reuse connection")
|
|
|
|
|
|
|
|
// NewSingleUseDialer returns a dialer that returns the given connection once
|
|
|
|
// and after that always fails with the ErrNoConnReuse error.
|
|
|
|
func NewSingleUseDialer(conn net.Conn) Dialer {
|
|
|
|
return &dialerSingleUse{conn: conn}
|
|
|
|
}
|
|
|
|
|
2021-09-06 22:14:49 +02:00
|
|
|
// dialerSingleUse is the Dialer returned by NewSingleDialer.
|
2021-09-06 14:12:30 +02:00
|
|
|
type dialerSingleUse struct {
|
|
|
|
sync.Mutex
|
|
|
|
conn net.Conn
|
|
|
|
}
|
|
|
|
|
|
|
|
var _ Dialer = &dialerSingleUse{}
|
|
|
|
|
|
|
|
func (s *dialerSingleUse) DialContext(ctx context.Context, network string, addr string) (net.Conn, error) {
|
|
|
|
defer s.Unlock()
|
|
|
|
s.Lock()
|
|
|
|
if s.conn == nil {
|
|
|
|
return nil, ErrNoConnReuse
|
|
|
|
}
|
|
|
|
var conn net.Conn
|
|
|
|
conn, s.conn = s.conn, nil
|
|
|
|
return conn, nil
|
|
|
|
}
|
|
|
|
|
|
|
|
func (s *dialerSingleUse) CloseIdleConnections() {
|
2021-09-08 21:19:51 +02:00
|
|
|
// nothing to do
|
2021-09-06 14:12:30 +02:00
|
|
|
}
|
2021-09-07 19:56:42 +02:00
|
|
|
|
|
|
|
// dialerErrWrapper is a dialer that performs error wrapping. The connection
|
|
|
|
// returned by the DialContext function will also perform error wrapping.
|
|
|
|
type dialerErrWrapper struct {
|
|
|
|
Dialer
|
|
|
|
}
|
|
|
|
|
|
|
|
var _ Dialer = &dialerErrWrapper{}
|
|
|
|
|
|
|
|
func (d *dialerErrWrapper) DialContext(ctx context.Context, network, address string) (net.Conn, error) {
|
|
|
|
conn, err := d.Dialer.DialContext(ctx, network, address)
|
|
|
|
if err != nil {
|
2021-09-08 21:19:51 +02:00
|
|
|
return nil, errorsx.NewErrWrapper(
|
|
|
|
errorsx.ClassifyGenericError, errorsx.ConnectOperation, err)
|
2021-09-07 19:56:42 +02:00
|
|
|
}
|
|
|
|
return &dialerErrWrapperConn{Conn: conn}, nil
|
|
|
|
}
|
|
|
|
|
|
|
|
// dialerErrWrapperConn is a net.Conn that performs error wrapping.
|
|
|
|
type dialerErrWrapperConn struct {
|
|
|
|
net.Conn
|
|
|
|
}
|
|
|
|
|
|
|
|
var _ net.Conn = &dialerErrWrapperConn{}
|
|
|
|
|
|
|
|
func (c *dialerErrWrapperConn) Read(b []byte) (int, error) {
|
|
|
|
count, err := c.Conn.Read(b)
|
|
|
|
if err != nil {
|
2021-09-08 21:19:51 +02:00
|
|
|
return 0, errorsx.NewErrWrapper(
|
|
|
|
errorsx.ClassifyGenericError, errorsx.ReadOperation, err)
|
2021-09-07 19:56:42 +02:00
|
|
|
}
|
|
|
|
return count, nil
|
|
|
|
}
|
|
|
|
|
|
|
|
func (c *dialerErrWrapperConn) Write(b []byte) (int, error) {
|
|
|
|
count, err := c.Conn.Write(b)
|
|
|
|
if err != nil {
|
2021-09-08 21:19:51 +02:00
|
|
|
return 0, errorsx.NewErrWrapper(
|
|
|
|
errorsx.ClassifyGenericError, errorsx.WriteOperation, err)
|
2021-09-07 19:56:42 +02:00
|
|
|
}
|
|
|
|
return count, nil
|
|
|
|
}
|
|
|
|
|
|
|
|
func (c *dialerErrWrapperConn) Close() error {
|
|
|
|
err := c.Conn.Close()
|
|
|
|
if err != nil {
|
2021-09-08 21:19:51 +02:00
|
|
|
return errorsx.NewErrWrapper(
|
|
|
|
errorsx.ClassifyGenericError, errorsx.CloseOperation, err)
|
2021-09-07 19:56:42 +02:00
|
|
|
}
|
|
|
|
return nil
|
|
|
|
}
|
2021-09-08 14:46:17 +02:00
|
|
|
|
|
|
|
// ErrNoDialer indicates that no dialer is configured.
|
|
|
|
var ErrNoDialer = errors.New("no configured dialer")
|
|
|
|
|
|
|
|
// NewNullDialer returns a dialer that always fails.
|
|
|
|
func NewNullDialer() Dialer {
|
|
|
|
return &nullDialer{}
|
|
|
|
}
|
|
|
|
|
|
|
|
type nullDialer struct{}
|
|
|
|
|
|
|
|
var _ Dialer = &nullDialer{}
|
|
|
|
|
|
|
|
func (*nullDialer) DialContext(ctx context.Context, network, address string) (net.Conn, error) {
|
|
|
|
return nil, ErrNoDialer
|
|
|
|
}
|
|
|
|
|
|
|
|
func (*nullDialer) CloseIdleConnections() {
|
|
|
|
// nothing to do
|
|
|
|
}
|