refactor: start pivoting netx (#396)
What do I mean by pivoting? Netx is currently organized by row:
```
| dialer | quicdialer | resolver | ...
saving | | | | ...
errorwrapping | | | | ...
logging | | | | ...
mocking/sys | | | | ...
```
Every row needs to implement saving, errorwrapping, logging, mocking (or
adapting to the system or to some underlying library).
This causes cross package dependencies and, in turn, complexity. For
example, we need the `trace` package for supporting saving.
And `dialer`, `quickdialer`, et al. need to depend on such a package.
The same goes for errorwrapping.
This arrangement further complicates testing. For example, I am
currently working on https://github.com/ooni/probe/issues/1505 and
I realize it need to repeat integration tests in multiple places.
Let's say instead we pivot the above matrix as follows:
```
| saving | errorwrapping | logging | ...
dialer | | | | ...
quicdialer | | | | ...
logging | | | | ...
mocking/sys | | | | ...
...
```
In this way, now every row contains everything related to a specific
action to perform. We can now share code without relying on extra
support packages. What's more, we can write tests and, judding from
the way in which things are made, it seems we only need integration
testing in `errorwrapping` because it's where data quality matters
whereas, in all other cases, unit testing is fine.
I am going, therefore, to proceed with these changes and "pivot"
`netx`. Hopefully, it won't be too painful.
2021-06-23 15:53:12 +02:00
|
|
|
package netxlite
|
|
|
|
|
|
|
|
import (
|
|
|
|
"context"
|
2021-09-05 20:41:46 +02:00
|
|
|
"errors"
|
2021-09-27 16:48:46 +02:00
|
|
|
"fmt"
|
refactor: start pivoting netx (#396)
What do I mean by pivoting? Netx is currently organized by row:
```
| dialer | quicdialer | resolver | ...
saving | | | | ...
errorwrapping | | | | ...
logging | | | | ...
mocking/sys | | | | ...
```
Every row needs to implement saving, errorwrapping, logging, mocking (or
adapting to the system or to some underlying library).
This causes cross package dependencies and, in turn, complexity. For
example, we need the `trace` package for supporting saving.
And `dialer`, `quickdialer`, et al. need to depend on such a package.
The same goes for errorwrapping.
This arrangement further complicates testing. For example, I am
currently working on https://github.com/ooni/probe/issues/1505 and
I realize it need to repeat integration tests in multiple places.
Let's say instead we pivot the above matrix as follows:
```
| saving | errorwrapping | logging | ...
dialer | | | | ...
quicdialer | | | | ...
logging | | | | ...
mocking/sys | | | | ...
...
```
In this way, now every row contains everything related to a specific
action to perform. We can now share code without relying on extra
support packages. What's more, we can write tests and, judding from
the way in which things are made, it seems we only need integration
testing in `errorwrapping` because it's where data quality matters
whereas, in all other cases, unit testing is fine.
I am going, therefore, to proceed with these changes and "pivot"
`netx`. Hopefully, it won't be too painful.
2021-06-23 15:53:12 +02:00
|
|
|
"net"
|
|
|
|
"time"
|
2021-08-17 10:29:06 +02:00
|
|
|
|
2021-09-27 23:09:41 +02:00
|
|
|
"github.com/ooni/probe-cli/v3/internal/netxlite/dnsx"
|
2021-09-07 19:56:42 +02:00
|
|
|
"github.com/ooni/probe-cli/v3/internal/netxlite/errorsx"
|
2021-08-17 10:29:06 +02:00
|
|
|
"golang.org/x/net/idna"
|
refactor: start pivoting netx (#396)
What do I mean by pivoting? Netx is currently organized by row:
```
| dialer | quicdialer | resolver | ...
saving | | | | ...
errorwrapping | | | | ...
logging | | | | ...
mocking/sys | | | | ...
```
Every row needs to implement saving, errorwrapping, logging, mocking (or
adapting to the system or to some underlying library).
This causes cross package dependencies and, in turn, complexity. For
example, we need the `trace` package for supporting saving.
And `dialer`, `quickdialer`, et al. need to depend on such a package.
The same goes for errorwrapping.
This arrangement further complicates testing. For example, I am
currently working on https://github.com/ooni/probe/issues/1505 and
I realize it need to repeat integration tests in multiple places.
Let's say instead we pivot the above matrix as follows:
```
| saving | errorwrapping | logging | ...
dialer | | | | ...
quicdialer | | | | ...
logging | | | | ...
mocking/sys | | | | ...
...
```
In this way, now every row contains everything related to a specific
action to perform. We can now share code without relying on extra
support packages. What's more, we can write tests and, judding from
the way in which things are made, it seems we only need integration
testing in `errorwrapping` because it's where data quality matters
whereas, in all other cases, unit testing is fine.
I am going, therefore, to proceed with these changes and "pivot"
`netx`. Hopefully, it won't be too painful.
2021-06-23 15:53:12 +02:00
|
|
|
)
|
|
|
|
|
2021-09-27 23:09:41 +02:00
|
|
|
// HTTPSSvc is the type returned for HTTPSSvc queries.
|
|
|
|
type HTTPSSvc = dnsx.HTTPSSvc
|
|
|
|
|
refactor: start pivoting netx (#396)
What do I mean by pivoting? Netx is currently organized by row:
```
| dialer | quicdialer | resolver | ...
saving | | | | ...
errorwrapping | | | | ...
logging | | | | ...
mocking/sys | | | | ...
```
Every row needs to implement saving, errorwrapping, logging, mocking (or
adapting to the system or to some underlying library).
This causes cross package dependencies and, in turn, complexity. For
example, we need the `trace` package for supporting saving.
And `dialer`, `quickdialer`, et al. need to depend on such a package.
The same goes for errorwrapping.
This arrangement further complicates testing. For example, I am
currently working on https://github.com/ooni/probe/issues/1505 and
I realize it need to repeat integration tests in multiple places.
Let's say instead we pivot the above matrix as follows:
```
| saving | errorwrapping | logging | ...
dialer | | | | ...
quicdialer | | | | ...
logging | | | | ...
mocking/sys | | | | ...
...
```
In this way, now every row contains everything related to a specific
action to perform. We can now share code without relying on extra
support packages. What's more, we can write tests and, judding from
the way in which things are made, it seems we only need integration
testing in `errorwrapping` because it's where data quality matters
whereas, in all other cases, unit testing is fine.
I am going, therefore, to proceed with these changes and "pivot"
`netx`. Hopefully, it won't be too painful.
2021-06-23 15:53:12 +02:00
|
|
|
// Resolver performs domain name resolutions.
|
|
|
|
type Resolver interface {
|
|
|
|
// LookupHost behaves like net.Resolver.LookupHost.
|
|
|
|
LookupHost(ctx context.Context, hostname string) (addrs []string, err error)
|
2021-09-05 18:03:50 +02:00
|
|
|
|
|
|
|
// Network returns the resolver type (e.g., system, dot, doh).
|
|
|
|
Network() string
|
|
|
|
|
|
|
|
// Address returns the resolver address (e.g., 8.8.8.8:53).
|
|
|
|
Address() string
|
|
|
|
|
|
|
|
// CloseIdleConnections closes idle connections, if any.
|
|
|
|
CloseIdleConnections()
|
2021-09-27 23:09:41 +02:00
|
|
|
|
|
|
|
// LookupHTTPS issues a single HTTPS query for
|
|
|
|
// a domain without any retry mechanism whatsoever.
|
|
|
|
LookupHTTPS(
|
|
|
|
ctx context.Context, domain string) (*HTTPSSvc, error)
|
2021-09-05 18:03:50 +02:00
|
|
|
}
|
|
|
|
|
2021-09-27 23:09:41 +02:00
|
|
|
// ErrNoDNSTransport indicates that the requested Resolver operation
|
|
|
|
// cannot be performed because we're using the "system" resolver.
|
|
|
|
var ErrNoDNSTransport = errors.New("operation requires a DNS transport")
|
|
|
|
|
2021-09-27 12:00:43 +02:00
|
|
|
// NewResolverStdlib creates a new Resolver by combining
|
|
|
|
// WrapResolver with an internal "system" resolver type that
|
|
|
|
// adds extra functionality to net.Resolver.
|
|
|
|
func NewResolverStdlib(logger Logger) Resolver {
|
|
|
|
return WrapResolver(logger, &resolverSystem{})
|
|
|
|
}
|
|
|
|
|
|
|
|
// WrapResolver creates a new resolver that wraps an
|
|
|
|
// existing resolver to add these properties:
|
2021-09-08 22:48:10 +02:00
|
|
|
//
|
|
|
|
// 1. handles IDNA;
|
|
|
|
//
|
|
|
|
// 2. performs logging;
|
|
|
|
//
|
|
|
|
// 3. short-circuits IP addresses like getaddrinfo does (i.e.,
|
|
|
|
// resolving "1.1.1.1" yields []string{"1.1.1.1"};
|
|
|
|
//
|
|
|
|
// 4. wraps errors;
|
|
|
|
//
|
|
|
|
// 5. enforces reasonable timeouts (
|
|
|
|
// see https://github.com/ooni/probe/issues/1726).
|
2021-09-27 12:00:43 +02:00
|
|
|
func WrapResolver(logger Logger, resolver Resolver) Resolver {
|
2021-09-05 18:03:50 +02:00
|
|
|
return &resolverIDNA{
|
|
|
|
Resolver: &resolverLogger{
|
2021-09-05 20:12:05 +02:00
|
|
|
Resolver: &resolverShortCircuitIPAddr{
|
2021-09-07 19:56:42 +02:00
|
|
|
Resolver: &resolverErrWrapper{
|
2021-09-27 12:00:43 +02:00
|
|
|
Resolver: resolver,
|
2021-09-07 19:56:42 +02:00
|
|
|
},
|
2021-09-05 20:12:05 +02:00
|
|
|
},
|
2021-09-05 20:41:46 +02:00
|
|
|
Logger: logger,
|
2021-09-05 18:03:50 +02:00
|
|
|
},
|
|
|
|
}
|
refactor: start pivoting netx (#396)
What do I mean by pivoting? Netx is currently organized by row:
```
| dialer | quicdialer | resolver | ...
saving | | | | ...
errorwrapping | | | | ...
logging | | | | ...
mocking/sys | | | | ...
```
Every row needs to implement saving, errorwrapping, logging, mocking (or
adapting to the system or to some underlying library).
This causes cross package dependencies and, in turn, complexity. For
example, we need the `trace` package for supporting saving.
And `dialer`, `quickdialer`, et al. need to depend on such a package.
The same goes for errorwrapping.
This arrangement further complicates testing. For example, I am
currently working on https://github.com/ooni/probe/issues/1505 and
I realize it need to repeat integration tests in multiple places.
Let's say instead we pivot the above matrix as follows:
```
| saving | errorwrapping | logging | ...
dialer | | | | ...
quicdialer | | | | ...
logging | | | | ...
mocking/sys | | | | ...
...
```
In this way, now every row contains everything related to a specific
action to perform. We can now share code without relying on extra
support packages. What's more, we can write tests and, judding from
the way in which things are made, it seems we only need integration
testing in `errorwrapping` because it's where data quality matters
whereas, in all other cases, unit testing is fine.
I am going, therefore, to proceed with these changes and "pivot"
`netx`. Hopefully, it won't be too painful.
2021-06-23 15:53:12 +02:00
|
|
|
}
|
|
|
|
|
2021-09-05 14:49:38 +02:00
|
|
|
// resolverSystem is the system resolver.
|
2021-09-05 18:50:05 +02:00
|
|
|
type resolverSystem struct {
|
|
|
|
testableTimeout time.Duration
|
|
|
|
testableLookupHost func(ctx context.Context, domain string) ([]string, error)
|
|
|
|
}
|
refactor: start pivoting netx (#396)
What do I mean by pivoting? Netx is currently organized by row:
```
| dialer | quicdialer | resolver | ...
saving | | | | ...
errorwrapping | | | | ...
logging | | | | ...
mocking/sys | | | | ...
```
Every row needs to implement saving, errorwrapping, logging, mocking (or
adapting to the system or to some underlying library).
This causes cross package dependencies and, in turn, complexity. For
example, we need the `trace` package for supporting saving.
And `dialer`, `quickdialer`, et al. need to depend on such a package.
The same goes for errorwrapping.
This arrangement further complicates testing. For example, I am
currently working on https://github.com/ooni/probe/issues/1505 and
I realize it need to repeat integration tests in multiple places.
Let's say instead we pivot the above matrix as follows:
```
| saving | errorwrapping | logging | ...
dialer | | | | ...
quicdialer | | | | ...
logging | | | | ...
mocking/sys | | | | ...
...
```
In this way, now every row contains everything related to a specific
action to perform. We can now share code without relying on extra
support packages. What's more, we can write tests and, judding from
the way in which things are made, it seems we only need integration
testing in `errorwrapping` because it's where data quality matters
whereas, in all other cases, unit testing is fine.
I am going, therefore, to proceed with these changes and "pivot"
`netx`. Hopefully, it won't be too painful.
2021-06-23 15:53:12 +02:00
|
|
|
|
2021-09-05 14:49:38 +02:00
|
|
|
var _ Resolver = &resolverSystem{}
|
refactor: start pivoting netx (#396)
What do I mean by pivoting? Netx is currently organized by row:
```
| dialer | quicdialer | resolver | ...
saving | | | | ...
errorwrapping | | | | ...
logging | | | | ...
mocking/sys | | | | ...
```
Every row needs to implement saving, errorwrapping, logging, mocking (or
adapting to the system or to some underlying library).
This causes cross package dependencies and, in turn, complexity. For
example, we need the `trace` package for supporting saving.
And `dialer`, `quickdialer`, et al. need to depend on such a package.
The same goes for errorwrapping.
This arrangement further complicates testing. For example, I am
currently working on https://github.com/ooni/probe/issues/1505 and
I realize it need to repeat integration tests in multiple places.
Let's say instead we pivot the above matrix as follows:
```
| saving | errorwrapping | logging | ...
dialer | | | | ...
quicdialer | | | | ...
logging | | | | ...
mocking/sys | | | | ...
...
```
In this way, now every row contains everything related to a specific
action to perform. We can now share code without relying on extra
support packages. What's more, we can write tests and, judding from
the way in which things are made, it seems we only need integration
testing in `errorwrapping` because it's where data quality matters
whereas, in all other cases, unit testing is fine.
I am going, therefore, to proceed with these changes and "pivot"
`netx`. Hopefully, it won't be too painful.
2021-06-23 15:53:12 +02:00
|
|
|
|
2021-09-05 14:49:38 +02:00
|
|
|
func (r *resolverSystem) LookupHost(ctx context.Context, hostname string) ([]string, error) {
|
2021-09-05 18:50:05 +02:00
|
|
|
// This code forces adding a shorter timeout to the domain name
|
|
|
|
// resolutions when using the system resolver. We have seen cases
|
|
|
|
// in which such a timeout becomes too large. One such case is
|
|
|
|
// described in https://github.com/ooni/probe/issues/1726.
|
|
|
|
addrsch, errch := make(chan []string, 1), make(chan error, 1)
|
|
|
|
ctx, cancel := context.WithTimeout(ctx, r.timeout())
|
|
|
|
defer cancel()
|
|
|
|
go func() {
|
|
|
|
addrs, err := r.lookupHost()(ctx, hostname)
|
|
|
|
if err != nil {
|
|
|
|
errch <- err
|
|
|
|
return
|
|
|
|
}
|
|
|
|
addrsch <- addrs
|
|
|
|
}()
|
|
|
|
select {
|
|
|
|
case <-ctx.Done():
|
|
|
|
return nil, ctx.Err()
|
|
|
|
case addrs := <-addrsch:
|
|
|
|
return addrs, nil
|
|
|
|
case err := <-errch:
|
|
|
|
return nil, err
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
func (r *resolverSystem) timeout() time.Duration {
|
|
|
|
if r.testableTimeout > 0 {
|
|
|
|
return r.testableTimeout
|
|
|
|
}
|
|
|
|
return 15 * time.Second
|
|
|
|
}
|
|
|
|
|
|
|
|
func (r *resolverSystem) lookupHost() func(ctx context.Context, domain string) ([]string, error) {
|
|
|
|
if r.testableLookupHost != nil {
|
|
|
|
return r.testableLookupHost
|
|
|
|
}
|
|
|
|
return net.DefaultResolver.LookupHost
|
refactor: start pivoting netx (#396)
What do I mean by pivoting? Netx is currently organized by row:
```
| dialer | quicdialer | resolver | ...
saving | | | | ...
errorwrapping | | | | ...
logging | | | | ...
mocking/sys | | | | ...
```
Every row needs to implement saving, errorwrapping, logging, mocking (or
adapting to the system or to some underlying library).
This causes cross package dependencies and, in turn, complexity. For
example, we need the `trace` package for supporting saving.
And `dialer`, `quickdialer`, et al. need to depend on such a package.
The same goes for errorwrapping.
This arrangement further complicates testing. For example, I am
currently working on https://github.com/ooni/probe/issues/1505 and
I realize it need to repeat integration tests in multiple places.
Let's say instead we pivot the above matrix as follows:
```
| saving | errorwrapping | logging | ...
dialer | | | | ...
quicdialer | | | | ...
logging | | | | ...
mocking/sys | | | | ...
...
```
In this way, now every row contains everything related to a specific
action to perform. We can now share code without relying on extra
support packages. What's more, we can write tests and, judding from
the way in which things are made, it seems we only need integration
testing in `errorwrapping` because it's where data quality matters
whereas, in all other cases, unit testing is fine.
I am going, therefore, to proceed with these changes and "pivot"
`netx`. Hopefully, it won't be too painful.
2021-06-23 15:53:12 +02:00
|
|
|
}
|
|
|
|
|
2021-09-05 14:49:38 +02:00
|
|
|
func (r *resolverSystem) Network() string {
|
refactor: start pivoting netx (#396)
What do I mean by pivoting? Netx is currently organized by row:
```
| dialer | quicdialer | resolver | ...
saving | | | | ...
errorwrapping | | | | ...
logging | | | | ...
mocking/sys | | | | ...
```
Every row needs to implement saving, errorwrapping, logging, mocking (or
adapting to the system or to some underlying library).
This causes cross package dependencies and, in turn, complexity. For
example, we need the `trace` package for supporting saving.
And `dialer`, `quickdialer`, et al. need to depend on such a package.
The same goes for errorwrapping.
This arrangement further complicates testing. For example, I am
currently working on https://github.com/ooni/probe/issues/1505 and
I realize it need to repeat integration tests in multiple places.
Let's say instead we pivot the above matrix as follows:
```
| saving | errorwrapping | logging | ...
dialer | | | | ...
quicdialer | | | | ...
logging | | | | ...
mocking/sys | | | | ...
...
```
In this way, now every row contains everything related to a specific
action to perform. We can now share code without relying on extra
support packages. What's more, we can write tests and, judding from
the way in which things are made, it seems we only need integration
testing in `errorwrapping` because it's where data quality matters
whereas, in all other cases, unit testing is fine.
I am going, therefore, to proceed with these changes and "pivot"
`netx`. Hopefully, it won't be too painful.
2021-06-23 15:53:12 +02:00
|
|
|
return "system"
|
|
|
|
}
|
|
|
|
|
2021-09-05 14:49:38 +02:00
|
|
|
func (r *resolverSystem) Address() string {
|
refactor: start pivoting netx (#396)
What do I mean by pivoting? Netx is currently organized by row:
```
| dialer | quicdialer | resolver | ...
saving | | | | ...
errorwrapping | | | | ...
logging | | | | ...
mocking/sys | | | | ...
```
Every row needs to implement saving, errorwrapping, logging, mocking (or
adapting to the system or to some underlying library).
This causes cross package dependencies and, in turn, complexity. For
example, we need the `trace` package for supporting saving.
And `dialer`, `quickdialer`, et al. need to depend on such a package.
The same goes for errorwrapping.
This arrangement further complicates testing. For example, I am
currently working on https://github.com/ooni/probe/issues/1505 and
I realize it need to repeat integration tests in multiple places.
Let's say instead we pivot the above matrix as follows:
```
| saving | errorwrapping | logging | ...
dialer | | | | ...
quicdialer | | | | ...
logging | | | | ...
mocking/sys | | | | ...
...
```
In this way, now every row contains everything related to a specific
action to perform. We can now share code without relying on extra
support packages. What's more, we can write tests and, judding from
the way in which things are made, it seems we only need integration
testing in `errorwrapping` because it's where data quality matters
whereas, in all other cases, unit testing is fine.
I am going, therefore, to proceed with these changes and "pivot"
`netx`. Hopefully, it won't be too painful.
2021-06-23 15:53:12 +02:00
|
|
|
return ""
|
|
|
|
}
|
|
|
|
|
2021-09-05 18:03:50 +02:00
|
|
|
func (r *resolverSystem) CloseIdleConnections() {
|
2021-09-08 22:48:10 +02:00
|
|
|
// nothing to do
|
2021-09-05 18:03:50 +02:00
|
|
|
}
|
|
|
|
|
2021-09-27 23:09:41 +02:00
|
|
|
func (r *resolverSystem) LookupHTTPS(
|
|
|
|
ctx context.Context, domain string) (*HTTPSSvc, error) {
|
|
|
|
return nil, ErrNoDNSTransport
|
|
|
|
}
|
|
|
|
|
2021-09-05 14:49:38 +02:00
|
|
|
// resolverLogger is a resolver that emits events
|
|
|
|
type resolverLogger struct {
|
refactor: start pivoting netx (#396)
What do I mean by pivoting? Netx is currently organized by row:
```
| dialer | quicdialer | resolver | ...
saving | | | | ...
errorwrapping | | | | ...
logging | | | | ...
mocking/sys | | | | ...
```
Every row needs to implement saving, errorwrapping, logging, mocking (or
adapting to the system or to some underlying library).
This causes cross package dependencies and, in turn, complexity. For
example, we need the `trace` package for supporting saving.
And `dialer`, `quickdialer`, et al. need to depend on such a package.
The same goes for errorwrapping.
This arrangement further complicates testing. For example, I am
currently working on https://github.com/ooni/probe/issues/1505 and
I realize it need to repeat integration tests in multiple places.
Let's say instead we pivot the above matrix as follows:
```
| saving | errorwrapping | logging | ...
dialer | | | | ...
quicdialer | | | | ...
logging | | | | ...
mocking/sys | | | | ...
...
```
In this way, now every row contains everything related to a specific
action to perform. We can now share code without relying on extra
support packages. What's more, we can write tests and, judding from
the way in which things are made, it seems we only need integration
testing in `errorwrapping` because it's where data quality matters
whereas, in all other cases, unit testing is fine.
I am going, therefore, to proceed with these changes and "pivot"
`netx`. Hopefully, it won't be too painful.
2021-06-23 15:53:12 +02:00
|
|
|
Resolver
|
|
|
|
Logger Logger
|
|
|
|
}
|
|
|
|
|
2021-09-05 14:49:38 +02:00
|
|
|
var _ Resolver = &resolverLogger{}
|
refactor: start pivoting netx (#396)
What do I mean by pivoting? Netx is currently organized by row:
```
| dialer | quicdialer | resolver | ...
saving | | | | ...
errorwrapping | | | | ...
logging | | | | ...
mocking/sys | | | | ...
```
Every row needs to implement saving, errorwrapping, logging, mocking (or
adapting to the system or to some underlying library).
This causes cross package dependencies and, in turn, complexity. For
example, we need the `trace` package for supporting saving.
And `dialer`, `quickdialer`, et al. need to depend on such a package.
The same goes for errorwrapping.
This arrangement further complicates testing. For example, I am
currently working on https://github.com/ooni/probe/issues/1505 and
I realize it need to repeat integration tests in multiple places.
Let's say instead we pivot the above matrix as follows:
```
| saving | errorwrapping | logging | ...
dialer | | | | ...
quicdialer | | | | ...
logging | | | | ...
mocking/sys | | | | ...
...
```
In this way, now every row contains everything related to a specific
action to perform. We can now share code without relying on extra
support packages. What's more, we can write tests and, judding from
the way in which things are made, it seems we only need integration
testing in `errorwrapping` because it's where data quality matters
whereas, in all other cases, unit testing is fine.
I am going, therefore, to proceed with these changes and "pivot"
`netx`. Hopefully, it won't be too painful.
2021-06-23 15:53:12 +02:00
|
|
|
|
2021-09-05 14:49:38 +02:00
|
|
|
func (r *resolverLogger) LookupHost(ctx context.Context, hostname string) ([]string, error) {
|
2021-09-27 16:48:46 +02:00
|
|
|
prefix := fmt.Sprintf("resolve[A,AAAA] %s with %s (%s)", hostname, r.Network(), r.Address())
|
|
|
|
r.Logger.Debugf("%s...", prefix)
|
refactor: start pivoting netx (#396)
What do I mean by pivoting? Netx is currently organized by row:
```
| dialer | quicdialer | resolver | ...
saving | | | | ...
errorwrapping | | | | ...
logging | | | | ...
mocking/sys | | | | ...
```
Every row needs to implement saving, errorwrapping, logging, mocking (or
adapting to the system or to some underlying library).
This causes cross package dependencies and, in turn, complexity. For
example, we need the `trace` package for supporting saving.
And `dialer`, `quickdialer`, et al. need to depend on such a package.
The same goes for errorwrapping.
This arrangement further complicates testing. For example, I am
currently working on https://github.com/ooni/probe/issues/1505 and
I realize it need to repeat integration tests in multiple places.
Let's say instead we pivot the above matrix as follows:
```
| saving | errorwrapping | logging | ...
dialer | | | | ...
quicdialer | | | | ...
logging | | | | ...
mocking/sys | | | | ...
...
```
In this way, now every row contains everything related to a specific
action to perform. We can now share code without relying on extra
support packages. What's more, we can write tests and, judding from
the way in which things are made, it seems we only need integration
testing in `errorwrapping` because it's where data quality matters
whereas, in all other cases, unit testing is fine.
I am going, therefore, to proceed with these changes and "pivot"
`netx`. Hopefully, it won't be too painful.
2021-06-23 15:53:12 +02:00
|
|
|
start := time.Now()
|
|
|
|
addrs, err := r.Resolver.LookupHost(ctx, hostname)
|
2021-06-23 17:00:44 +02:00
|
|
|
elapsed := time.Since(start)
|
|
|
|
if err != nil {
|
2021-09-27 16:48:46 +02:00
|
|
|
r.Logger.Debugf("%s... %s in %s", prefix, err, elapsed)
|
2021-06-23 17:00:44 +02:00
|
|
|
return nil, err
|
|
|
|
}
|
2021-09-27 16:48:46 +02:00
|
|
|
r.Logger.Debugf("%s... %+v in %s", prefix, addrs, elapsed)
|
2021-06-23 17:00:44 +02:00
|
|
|
return addrs, nil
|
refactor: start pivoting netx (#396)
What do I mean by pivoting? Netx is currently organized by row:
```
| dialer | quicdialer | resolver | ...
saving | | | | ...
errorwrapping | | | | ...
logging | | | | ...
mocking/sys | | | | ...
```
Every row needs to implement saving, errorwrapping, logging, mocking (or
adapting to the system or to some underlying library).
This causes cross package dependencies and, in turn, complexity. For
example, we need the `trace` package for supporting saving.
And `dialer`, `quickdialer`, et al. need to depend on such a package.
The same goes for errorwrapping.
This arrangement further complicates testing. For example, I am
currently working on https://github.com/ooni/probe/issues/1505 and
I realize it need to repeat integration tests in multiple places.
Let's say instead we pivot the above matrix as follows:
```
| saving | errorwrapping | logging | ...
dialer | | | | ...
quicdialer | | | | ...
logging | | | | ...
mocking/sys | | | | ...
...
```
In this way, now every row contains everything related to a specific
action to perform. We can now share code without relying on extra
support packages. What's more, we can write tests and, judding from
the way in which things are made, it seems we only need integration
testing in `errorwrapping` because it's where data quality matters
whereas, in all other cases, unit testing is fine.
I am going, therefore, to proceed with these changes and "pivot"
`netx`. Hopefully, it won't be too painful.
2021-06-23 15:53:12 +02:00
|
|
|
}
|
|
|
|
|
2021-09-27 23:09:41 +02:00
|
|
|
func (r *resolverLogger) LookupHTTPS(
|
|
|
|
ctx context.Context, domain string) (*HTTPSSvc, error) {
|
|
|
|
prefix := fmt.Sprintf("resolve[HTTPS] %s with %s (%s)", domain, r.Network(), r.Address())
|
|
|
|
r.Logger.Debugf("%s...", prefix)
|
|
|
|
start := time.Now()
|
|
|
|
https, err := r.Resolver.LookupHTTPS(ctx, domain)
|
|
|
|
elapsed := time.Since(start)
|
|
|
|
if err != nil {
|
|
|
|
r.Logger.Debugf("%s... %s in %s", prefix, err, elapsed)
|
|
|
|
return nil, err
|
|
|
|
}
|
|
|
|
alpn := https.ALPN
|
|
|
|
a := https.IPv4
|
|
|
|
aaaa := https.IPv6
|
|
|
|
r.Logger.Debugf("%s... %+v %+v %+v in %s", prefix, alpn, a, aaaa, elapsed)
|
|
|
|
return https, nil
|
|
|
|
}
|
|
|
|
|
2021-09-05 14:49:38 +02:00
|
|
|
// resolverIDNA supports resolving Internationalized Domain Names.
|
2021-08-17 11:02:12 +02:00
|
|
|
//
|
2021-08-17 10:29:06 +02:00
|
|
|
// See RFC3492 for more information.
|
2021-09-05 14:49:38 +02:00
|
|
|
type resolverIDNA struct {
|
2021-08-17 10:29:06 +02:00
|
|
|
Resolver
|
|
|
|
}
|
|
|
|
|
2021-09-05 14:49:38 +02:00
|
|
|
func (r *resolverIDNA) LookupHost(ctx context.Context, hostname string) ([]string, error) {
|
2021-08-17 10:29:06 +02:00
|
|
|
host, err := idna.ToASCII(hostname)
|
|
|
|
if err != nil {
|
|
|
|
return nil, err
|
|
|
|
}
|
|
|
|
return r.Resolver.LookupHost(ctx, host)
|
|
|
|
}
|
2021-09-05 20:12:05 +02:00
|
|
|
|
2021-09-27 23:09:41 +02:00
|
|
|
func (r *resolverIDNA) LookupHTTPS(
|
|
|
|
ctx context.Context, domain string) (*HTTPSSvc, error) {
|
|
|
|
host, err := idna.ToASCII(domain)
|
|
|
|
if err != nil {
|
|
|
|
return nil, err
|
|
|
|
}
|
|
|
|
return r.Resolver.LookupHTTPS(ctx, host)
|
|
|
|
}
|
|
|
|
|
2021-09-05 20:12:05 +02:00
|
|
|
// resolverShortCircuitIPAddr recognizes when the input hostname is an
|
|
|
|
// IP address and returns it immediately to the caller.
|
|
|
|
type resolverShortCircuitIPAddr struct {
|
|
|
|
Resolver
|
|
|
|
}
|
|
|
|
|
|
|
|
func (r *resolverShortCircuitIPAddr) LookupHost(ctx context.Context, hostname string) ([]string, error) {
|
|
|
|
if net.ParseIP(hostname) != nil {
|
|
|
|
return []string{hostname}, nil
|
|
|
|
}
|
|
|
|
return r.Resolver.LookupHost(ctx, hostname)
|
|
|
|
}
|
2021-09-05 20:41:46 +02:00
|
|
|
|
|
|
|
// ErrNoResolver indicates you are using a dialer without a resolver.
|
|
|
|
var ErrNoResolver = errors.New("no configured resolver")
|
|
|
|
|
|
|
|
// nullResolver is a resolver that is not capable of resolving
|
|
|
|
// domain names to IP addresses and always returns ErrNoResolver.
|
|
|
|
type nullResolver struct{}
|
|
|
|
|
|
|
|
func (r *nullResolver) LookupHost(ctx context.Context, hostname string) (addrs []string, err error) {
|
|
|
|
return nil, ErrNoResolver
|
|
|
|
}
|
|
|
|
|
|
|
|
func (r *nullResolver) Network() string {
|
|
|
|
return "null"
|
|
|
|
}
|
|
|
|
|
|
|
|
func (r *nullResolver) Address() string {
|
|
|
|
return ""
|
|
|
|
}
|
|
|
|
|
|
|
|
func (r *nullResolver) CloseIdleConnections() {
|
2021-09-08 22:48:10 +02:00
|
|
|
// nothing to do
|
2021-09-05 20:41:46 +02:00
|
|
|
}
|
2021-09-07 19:56:42 +02:00
|
|
|
|
2021-09-27 23:09:41 +02:00
|
|
|
func (r *nullResolver) LookupHTTPS(
|
|
|
|
ctx context.Context, domain string) (*HTTPSSvc, error) {
|
|
|
|
return nil, ErrNoResolver
|
|
|
|
}
|
|
|
|
|
2021-09-07 19:56:42 +02:00
|
|
|
// resolverErrWrapper is a Resolver that knows about wrapping errors.
|
|
|
|
type resolverErrWrapper struct {
|
|
|
|
Resolver
|
|
|
|
}
|
|
|
|
|
|
|
|
var _ Resolver = &resolverErrWrapper{}
|
|
|
|
|
|
|
|
func (r *resolverErrWrapper) LookupHost(ctx context.Context, hostname string) ([]string, error) {
|
|
|
|
addrs, err := r.Resolver.LookupHost(ctx, hostname)
|
|
|
|
if err != nil {
|
2021-09-08 21:19:51 +02:00
|
|
|
return nil, errorsx.NewErrWrapper(
|
|
|
|
errorsx.ClassifyResolverError, errorsx.ResolveOperation, err)
|
2021-09-07 19:56:42 +02:00
|
|
|
}
|
|
|
|
return addrs, nil
|
|
|
|
}
|
2021-09-27 23:09:41 +02:00
|
|
|
|
|
|
|
func (r *resolverErrWrapper) LookupHTTPS(
|
|
|
|
ctx context.Context, domain string) (*HTTPSSvc, error) {
|
|
|
|
out, err := r.Resolver.LookupHTTPS(ctx, domain)
|
|
|
|
if err != nil {
|
|
|
|
return nil, errorsx.NewErrWrapper(
|
|
|
|
errorsx.ClassifyResolverError, errorsx.ResolveOperation, err)
|
|
|
|
}
|
|
|
|
return out, nil
|
|
|
|
}
|