refactor: start pivoting netx (#396)
What do I mean by pivoting? Netx is currently organized by row:
```
| dialer | quicdialer | resolver | ...
saving | | | | ...
errorwrapping | | | | ...
logging | | | | ...
mocking/sys | | | | ...
```
Every row needs to implement saving, errorwrapping, logging, mocking (or
adapting to the system or to some underlying library).
This causes cross package dependencies and, in turn, complexity. For
example, we need the `trace` package for supporting saving.
And `dialer`, `quickdialer`, et al. need to depend on such a package.
The same goes for errorwrapping.
This arrangement further complicates testing. For example, I am
currently working on https://github.com/ooni/probe/issues/1505 and
I realize it need to repeat integration tests in multiple places.
Let's say instead we pivot the above matrix as follows:
```
| saving | errorwrapping | logging | ...
dialer | | | | ...
quicdialer | | | | ...
logging | | | | ...
mocking/sys | | | | ...
...
```
In this way, now every row contains everything related to a specific
action to perform. We can now share code without relying on extra
support packages. What's more, we can write tests and, judding from
the way in which things are made, it seems we only need integration
testing in `errorwrapping` because it's where data quality matters
whereas, in all other cases, unit testing is fine.
I am going, therefore, to proceed with these changes and "pivot"
`netx`. Hopefully, it won't be too painful.
2021-06-23 15:53:12 +02:00
|
|
|
package netxlite
|
|
|
|
|
|
|
|
import (
|
|
|
|
"context"
|
2021-09-05 20:41:46 +02:00
|
|
|
"errors"
|
2021-09-27 16:48:46 +02:00
|
|
|
"fmt"
|
refactor: start pivoting netx (#396)
What do I mean by pivoting? Netx is currently organized by row:
```
| dialer | quicdialer | resolver | ...
saving | | | | ...
errorwrapping | | | | ...
logging | | | | ...
mocking/sys | | | | ...
```
Every row needs to implement saving, errorwrapping, logging, mocking (or
adapting to the system or to some underlying library).
This causes cross package dependencies and, in turn, complexity. For
example, we need the `trace` package for supporting saving.
And `dialer`, `quickdialer`, et al. need to depend on such a package.
The same goes for errorwrapping.
This arrangement further complicates testing. For example, I am
currently working on https://github.com/ooni/probe/issues/1505 and
I realize it need to repeat integration tests in multiple places.
Let's say instead we pivot the above matrix as follows:
```
| saving | errorwrapping | logging | ...
dialer | | | | ...
quicdialer | | | | ...
logging | | | | ...
mocking/sys | | | | ...
...
```
In this way, now every row contains everything related to a specific
action to perform. We can now share code without relying on extra
support packages. What's more, we can write tests and, judding from
the way in which things are made, it seems we only need integration
testing in `errorwrapping` because it's where data quality matters
whereas, in all other cases, unit testing is fine.
I am going, therefore, to proceed with these changes and "pivot"
`netx`. Hopefully, it won't be too painful.
2021-06-23 15:53:12 +02:00
|
|
|
"net"
|
|
|
|
"time"
|
2021-08-17 10:29:06 +02:00
|
|
|
|
2021-09-27 23:09:41 +02:00
|
|
|
"github.com/ooni/probe-cli/v3/internal/netxlite/dnsx"
|
2021-08-17 10:29:06 +02:00
|
|
|
"golang.org/x/net/idna"
|
refactor: start pivoting netx (#396)
What do I mean by pivoting? Netx is currently organized by row:
```
| dialer | quicdialer | resolver | ...
saving | | | | ...
errorwrapping | | | | ...
logging | | | | ...
mocking/sys | | | | ...
```
Every row needs to implement saving, errorwrapping, logging, mocking (or
adapting to the system or to some underlying library).
This causes cross package dependencies and, in turn, complexity. For
example, we need the `trace` package for supporting saving.
And `dialer`, `quickdialer`, et al. need to depend on such a package.
The same goes for errorwrapping.
This arrangement further complicates testing. For example, I am
currently working on https://github.com/ooni/probe/issues/1505 and
I realize it need to repeat integration tests in multiple places.
Let's say instead we pivot the above matrix as follows:
```
| saving | errorwrapping | logging | ...
dialer | | | | ...
quicdialer | | | | ...
logging | | | | ...
mocking/sys | | | | ...
...
```
In this way, now every row contains everything related to a specific
action to perform. We can now share code without relying on extra
support packages. What's more, we can write tests and, judding from
the way in which things are made, it seems we only need integration
testing in `errorwrapping` because it's where data quality matters
whereas, in all other cases, unit testing is fine.
I am going, therefore, to proceed with these changes and "pivot"
`netx`. Hopefully, it won't be too painful.
2021-06-23 15:53:12 +02:00
|
|
|
)
|
|
|
|
|
2021-09-29 20:21:25 +02:00
|
|
|
// HTTPSSvc is the type returned for HTTPS queries.
|
2021-09-27 23:09:41 +02:00
|
|
|
type HTTPSSvc = dnsx.HTTPSSvc
|
|
|
|
|
refactor: start pivoting netx (#396)
What do I mean by pivoting? Netx is currently organized by row:
```
| dialer | quicdialer | resolver | ...
saving | | | | ...
errorwrapping | | | | ...
logging | | | | ...
mocking/sys | | | | ...
```
Every row needs to implement saving, errorwrapping, logging, mocking (or
adapting to the system or to some underlying library).
This causes cross package dependencies and, in turn, complexity. For
example, we need the `trace` package for supporting saving.
And `dialer`, `quickdialer`, et al. need to depend on such a package.
The same goes for errorwrapping.
This arrangement further complicates testing. For example, I am
currently working on https://github.com/ooni/probe/issues/1505 and
I realize it need to repeat integration tests in multiple places.
Let's say instead we pivot the above matrix as follows:
```
| saving | errorwrapping | logging | ...
dialer | | | | ...
quicdialer | | | | ...
logging | | | | ...
mocking/sys | | | | ...
...
```
In this way, now every row contains everything related to a specific
action to perform. We can now share code without relying on extra
support packages. What's more, we can write tests and, judding from
the way in which things are made, it seems we only need integration
testing in `errorwrapping` because it's where data quality matters
whereas, in all other cases, unit testing is fine.
I am going, therefore, to proceed with these changes and "pivot"
`netx`. Hopefully, it won't be too painful.
2021-06-23 15:53:12 +02:00
|
|
|
// Resolver performs domain name resolutions.
|
|
|
|
type Resolver interface {
|
|
|
|
// LookupHost behaves like net.Resolver.LookupHost.
|
|
|
|
LookupHost(ctx context.Context, hostname string) (addrs []string, err error)
|
2021-09-05 18:03:50 +02:00
|
|
|
|
|
|
|
// Network returns the resolver type (e.g., system, dot, doh).
|
|
|
|
Network() string
|
|
|
|
|
|
|
|
// Address returns the resolver address (e.g., 8.8.8.8:53).
|
|
|
|
Address() string
|
|
|
|
|
|
|
|
// CloseIdleConnections closes idle connections, if any.
|
|
|
|
CloseIdleConnections()
|
2021-09-27 23:09:41 +02:00
|
|
|
|
2021-09-29 20:21:25 +02:00
|
|
|
// LookupHTTPS issues an HTTPS query for a domain.
|
2021-09-27 23:09:41 +02:00
|
|
|
LookupHTTPS(
|
|
|
|
ctx context.Context, domain string) (*HTTPSSvc, error)
|
2021-09-05 18:03:50 +02:00
|
|
|
}
|
|
|
|
|
2021-09-29 20:21:25 +02:00
|
|
|
// ErrNoDNSTransport is the error returned when you attempt to perform
|
|
|
|
// a DNS operation that requires a custom DNSTransport (e.g., DNSOverHTTPS)
|
|
|
|
// but you are using the "system" resolver instead.
|
2021-09-27 23:09:41 +02:00
|
|
|
var ErrNoDNSTransport = errors.New("operation requires a DNS transport")
|
|
|
|
|
2021-09-29 20:21:25 +02:00
|
|
|
// NewResolverStdlib creates a new Resolver by combining WrapResolver
|
|
|
|
// with an internal "system" resolver type.
|
2021-09-27 12:00:43 +02:00
|
|
|
func NewResolverStdlib(logger Logger) Resolver {
|
|
|
|
return WrapResolver(logger, &resolverSystem{})
|
|
|
|
}
|
|
|
|
|
2021-09-29 20:21:25 +02:00
|
|
|
// NewResolverUDP creates a new Resolver using DNS-over-UDP.
|
|
|
|
//
|
|
|
|
// Arguments:
|
|
|
|
//
|
|
|
|
// - logger is the logger to use
|
|
|
|
//
|
|
|
|
// - dialer is the dialer to create and connect UDP conns
|
|
|
|
//
|
|
|
|
// - address is the server address (e.g., 1.1.1.1:53)
|
2021-09-28 18:15:38 +02:00
|
|
|
func NewResolverUDP(logger Logger, dialer Dialer, address string) Resolver {
|
|
|
|
return WrapResolver(logger, NewSerialResolver(
|
|
|
|
NewDNSOverUDP(dialer, address),
|
|
|
|
))
|
|
|
|
}
|
|
|
|
|
2021-09-27 12:00:43 +02:00
|
|
|
// WrapResolver creates a new resolver that wraps an
|
|
|
|
// existing resolver to add these properties:
|
2021-09-08 22:48:10 +02:00
|
|
|
//
|
|
|
|
// 1. handles IDNA;
|
|
|
|
//
|
|
|
|
// 2. performs logging;
|
|
|
|
//
|
|
|
|
// 3. short-circuits IP addresses like getaddrinfo does (i.e.,
|
|
|
|
// resolving "1.1.1.1" yields []string{"1.1.1.1"};
|
|
|
|
//
|
|
|
|
// 4. wraps errors;
|
|
|
|
//
|
|
|
|
// 5. enforces reasonable timeouts (
|
|
|
|
// see https://github.com/ooni/probe/issues/1726).
|
2021-09-29 20:21:25 +02:00
|
|
|
//
|
|
|
|
// This is a low-level factory. Use only if out of alternatives.
|
2021-09-27 12:00:43 +02:00
|
|
|
func WrapResolver(logger Logger, resolver Resolver) Resolver {
|
2021-09-05 18:03:50 +02:00
|
|
|
return &resolverIDNA{
|
|
|
|
Resolver: &resolverLogger{
|
2021-09-05 20:12:05 +02:00
|
|
|
Resolver: &resolverShortCircuitIPAddr{
|
2021-09-07 19:56:42 +02:00
|
|
|
Resolver: &resolverErrWrapper{
|
2021-09-27 12:00:43 +02:00
|
|
|
Resolver: resolver,
|
2021-09-07 19:56:42 +02:00
|
|
|
},
|
2021-09-05 20:12:05 +02:00
|
|
|
},
|
2021-09-05 20:41:46 +02:00
|
|
|
Logger: logger,
|
2021-09-05 18:03:50 +02:00
|
|
|
},
|
|
|
|
}
|
refactor: start pivoting netx (#396)
What do I mean by pivoting? Netx is currently organized by row:
```
| dialer | quicdialer | resolver | ...
saving | | | | ...
errorwrapping | | | | ...
logging | | | | ...
mocking/sys | | | | ...
```
Every row needs to implement saving, errorwrapping, logging, mocking (or
adapting to the system or to some underlying library).
This causes cross package dependencies and, in turn, complexity. For
example, we need the `trace` package for supporting saving.
And `dialer`, `quickdialer`, et al. need to depend on such a package.
The same goes for errorwrapping.
This arrangement further complicates testing. For example, I am
currently working on https://github.com/ooni/probe/issues/1505 and
I realize it need to repeat integration tests in multiple places.
Let's say instead we pivot the above matrix as follows:
```
| saving | errorwrapping | logging | ...
dialer | | | | ...
quicdialer | | | | ...
logging | | | | ...
mocking/sys | | | | ...
...
```
In this way, now every row contains everything related to a specific
action to perform. We can now share code without relying on extra
support packages. What's more, we can write tests and, judding from
the way in which things are made, it seems we only need integration
testing in `errorwrapping` because it's where data quality matters
whereas, in all other cases, unit testing is fine.
I am going, therefore, to proceed with these changes and "pivot"
`netx`. Hopefully, it won't be too painful.
2021-06-23 15:53:12 +02:00
|
|
|
}
|
|
|
|
|
2021-09-05 14:49:38 +02:00
|
|
|
// resolverSystem is the system resolver.
|
2021-09-05 18:50:05 +02:00
|
|
|
type resolverSystem struct {
|
|
|
|
testableTimeout time.Duration
|
|
|
|
testableLookupHost func(ctx context.Context, domain string) ([]string, error)
|
|
|
|
}
|
refactor: start pivoting netx (#396)
What do I mean by pivoting? Netx is currently organized by row:
```
| dialer | quicdialer | resolver | ...
saving | | | | ...
errorwrapping | | | | ...
logging | | | | ...
mocking/sys | | | | ...
```
Every row needs to implement saving, errorwrapping, logging, mocking (or
adapting to the system or to some underlying library).
This causes cross package dependencies and, in turn, complexity. For
example, we need the `trace` package for supporting saving.
And `dialer`, `quickdialer`, et al. need to depend on such a package.
The same goes for errorwrapping.
This arrangement further complicates testing. For example, I am
currently working on https://github.com/ooni/probe/issues/1505 and
I realize it need to repeat integration tests in multiple places.
Let's say instead we pivot the above matrix as follows:
```
| saving | errorwrapping | logging | ...
dialer | | | | ...
quicdialer | | | | ...
logging | | | | ...
mocking/sys | | | | ...
...
```
In this way, now every row contains everything related to a specific
action to perform. We can now share code without relying on extra
support packages. What's more, we can write tests and, judding from
the way in which things are made, it seems we only need integration
testing in `errorwrapping` because it's where data quality matters
whereas, in all other cases, unit testing is fine.
I am going, therefore, to proceed with these changes and "pivot"
`netx`. Hopefully, it won't be too painful.
2021-06-23 15:53:12 +02:00
|
|
|
|
2021-09-05 14:49:38 +02:00
|
|
|
var _ Resolver = &resolverSystem{}
|
refactor: start pivoting netx (#396)
What do I mean by pivoting? Netx is currently organized by row:
```
| dialer | quicdialer | resolver | ...
saving | | | | ...
errorwrapping | | | | ...
logging | | | | ...
mocking/sys | | | | ...
```
Every row needs to implement saving, errorwrapping, logging, mocking (or
adapting to the system or to some underlying library).
This causes cross package dependencies and, in turn, complexity. For
example, we need the `trace` package for supporting saving.
And `dialer`, `quickdialer`, et al. need to depend on such a package.
The same goes for errorwrapping.
This arrangement further complicates testing. For example, I am
currently working on https://github.com/ooni/probe/issues/1505 and
I realize it need to repeat integration tests in multiple places.
Let's say instead we pivot the above matrix as follows:
```
| saving | errorwrapping | logging | ...
dialer | | | | ...
quicdialer | | | | ...
logging | | | | ...
mocking/sys | | | | ...
...
```
In this way, now every row contains everything related to a specific
action to perform. We can now share code without relying on extra
support packages. What's more, we can write tests and, judding from
the way in which things are made, it seems we only need integration
testing in `errorwrapping` because it's where data quality matters
whereas, in all other cases, unit testing is fine.
I am going, therefore, to proceed with these changes and "pivot"
`netx`. Hopefully, it won't be too painful.
2021-06-23 15:53:12 +02:00
|
|
|
|
2021-09-05 14:49:38 +02:00
|
|
|
func (r *resolverSystem) LookupHost(ctx context.Context, hostname string) ([]string, error) {
|
2021-09-05 18:50:05 +02:00
|
|
|
// This code forces adding a shorter timeout to the domain name
|
|
|
|
// resolutions when using the system resolver. We have seen cases
|
|
|
|
// in which such a timeout becomes too large. One such case is
|
|
|
|
// described in https://github.com/ooni/probe/issues/1726.
|
|
|
|
addrsch, errch := make(chan []string, 1), make(chan error, 1)
|
|
|
|
ctx, cancel := context.WithTimeout(ctx, r.timeout())
|
|
|
|
defer cancel()
|
|
|
|
go func() {
|
|
|
|
addrs, err := r.lookupHost()(ctx, hostname)
|
|
|
|
if err != nil {
|
|
|
|
errch <- err
|
|
|
|
return
|
|
|
|
}
|
|
|
|
addrsch <- addrs
|
|
|
|
}()
|
|
|
|
select {
|
|
|
|
case <-ctx.Done():
|
|
|
|
return nil, ctx.Err()
|
|
|
|
case addrs := <-addrsch:
|
|
|
|
return addrs, nil
|
|
|
|
case err := <-errch:
|
|
|
|
return nil, err
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
func (r *resolverSystem) timeout() time.Duration {
|
|
|
|
if r.testableTimeout > 0 {
|
|
|
|
return r.testableTimeout
|
|
|
|
}
|
|
|
|
return 15 * time.Second
|
|
|
|
}
|
|
|
|
|
|
|
|
func (r *resolverSystem) lookupHost() func(ctx context.Context, domain string) ([]string, error) {
|
|
|
|
if r.testableLookupHost != nil {
|
|
|
|
return r.testableLookupHost
|
|
|
|
}
|
2021-11-02 12:04:40 +01:00
|
|
|
return TProxy.LookupHost
|
refactor: start pivoting netx (#396)
What do I mean by pivoting? Netx is currently organized by row:
```
| dialer | quicdialer | resolver | ...
saving | | | | ...
errorwrapping | | | | ...
logging | | | | ...
mocking/sys | | | | ...
```
Every row needs to implement saving, errorwrapping, logging, mocking (or
adapting to the system or to some underlying library).
This causes cross package dependencies and, in turn, complexity. For
example, we need the `trace` package for supporting saving.
And `dialer`, `quickdialer`, et al. need to depend on such a package.
The same goes for errorwrapping.
This arrangement further complicates testing. For example, I am
currently working on https://github.com/ooni/probe/issues/1505 and
I realize it need to repeat integration tests in multiple places.
Let's say instead we pivot the above matrix as follows:
```
| saving | errorwrapping | logging | ...
dialer | | | | ...
quicdialer | | | | ...
logging | | | | ...
mocking/sys | | | | ...
...
```
In this way, now every row contains everything related to a specific
action to perform. We can now share code without relying on extra
support packages. What's more, we can write tests and, judding from
the way in which things are made, it seems we only need integration
testing in `errorwrapping` because it's where data quality matters
whereas, in all other cases, unit testing is fine.
I am going, therefore, to proceed with these changes and "pivot"
`netx`. Hopefully, it won't be too painful.
2021-06-23 15:53:12 +02:00
|
|
|
}
|
|
|
|
|
2021-09-05 14:49:38 +02:00
|
|
|
func (r *resolverSystem) Network() string {
|
refactor: start pivoting netx (#396)
What do I mean by pivoting? Netx is currently organized by row:
```
| dialer | quicdialer | resolver | ...
saving | | | | ...
errorwrapping | | | | ...
logging | | | | ...
mocking/sys | | | | ...
```
Every row needs to implement saving, errorwrapping, logging, mocking (or
adapting to the system or to some underlying library).
This causes cross package dependencies and, in turn, complexity. For
example, we need the `trace` package for supporting saving.
And `dialer`, `quickdialer`, et al. need to depend on such a package.
The same goes for errorwrapping.
This arrangement further complicates testing. For example, I am
currently working on https://github.com/ooni/probe/issues/1505 and
I realize it need to repeat integration tests in multiple places.
Let's say instead we pivot the above matrix as follows:
```
| saving | errorwrapping | logging | ...
dialer | | | | ...
quicdialer | | | | ...
logging | | | | ...
mocking/sys | | | | ...
...
```
In this way, now every row contains everything related to a specific
action to perform. We can now share code without relying on extra
support packages. What's more, we can write tests and, judding from
the way in which things are made, it seems we only need integration
testing in `errorwrapping` because it's where data quality matters
whereas, in all other cases, unit testing is fine.
I am going, therefore, to proceed with these changes and "pivot"
`netx`. Hopefully, it won't be too painful.
2021-06-23 15:53:12 +02:00
|
|
|
return "system"
|
|
|
|
}
|
|
|
|
|
2021-09-05 14:49:38 +02:00
|
|
|
func (r *resolverSystem) Address() string {
|
refactor: start pivoting netx (#396)
What do I mean by pivoting? Netx is currently organized by row:
```
| dialer | quicdialer | resolver | ...
saving | | | | ...
errorwrapping | | | | ...
logging | | | | ...
mocking/sys | | | | ...
```
Every row needs to implement saving, errorwrapping, logging, mocking (or
adapting to the system or to some underlying library).
This causes cross package dependencies and, in turn, complexity. For
example, we need the `trace` package for supporting saving.
And `dialer`, `quickdialer`, et al. need to depend on such a package.
The same goes for errorwrapping.
This arrangement further complicates testing. For example, I am
currently working on https://github.com/ooni/probe/issues/1505 and
I realize it need to repeat integration tests in multiple places.
Let's say instead we pivot the above matrix as follows:
```
| saving | errorwrapping | logging | ...
dialer | | | | ...
quicdialer | | | | ...
logging | | | | ...
mocking/sys | | | | ...
...
```
In this way, now every row contains everything related to a specific
action to perform. We can now share code without relying on extra
support packages. What's more, we can write tests and, judding from
the way in which things are made, it seems we only need integration
testing in `errorwrapping` because it's where data quality matters
whereas, in all other cases, unit testing is fine.
I am going, therefore, to proceed with these changes and "pivot"
`netx`. Hopefully, it won't be too painful.
2021-06-23 15:53:12 +02:00
|
|
|
return ""
|
|
|
|
}
|
|
|
|
|
2021-09-05 18:03:50 +02:00
|
|
|
func (r *resolverSystem) CloseIdleConnections() {
|
2021-09-08 22:48:10 +02:00
|
|
|
// nothing to do
|
2021-09-05 18:03:50 +02:00
|
|
|
}
|
|
|
|
|
2021-09-27 23:09:41 +02:00
|
|
|
func (r *resolverSystem) LookupHTTPS(
|
|
|
|
ctx context.Context, domain string) (*HTTPSSvc, error) {
|
|
|
|
return nil, ErrNoDNSTransport
|
|
|
|
}
|
|
|
|
|
2021-09-05 14:49:38 +02:00
|
|
|
// resolverLogger is a resolver that emits events
|
|
|
|
type resolverLogger struct {
|
refactor: start pivoting netx (#396)
What do I mean by pivoting? Netx is currently organized by row:
```
| dialer | quicdialer | resolver | ...
saving | | | | ...
errorwrapping | | | | ...
logging | | | | ...
mocking/sys | | | | ...
```
Every row needs to implement saving, errorwrapping, logging, mocking (or
adapting to the system or to some underlying library).
This causes cross package dependencies and, in turn, complexity. For
example, we need the `trace` package for supporting saving.
And `dialer`, `quickdialer`, et al. need to depend on such a package.
The same goes for errorwrapping.
This arrangement further complicates testing. For example, I am
currently working on https://github.com/ooni/probe/issues/1505 and
I realize it need to repeat integration tests in multiple places.
Let's say instead we pivot the above matrix as follows:
```
| saving | errorwrapping | logging | ...
dialer | | | | ...
quicdialer | | | | ...
logging | | | | ...
mocking/sys | | | | ...
...
```
In this way, now every row contains everything related to a specific
action to perform. We can now share code without relying on extra
support packages. What's more, we can write tests and, judding from
the way in which things are made, it seems we only need integration
testing in `errorwrapping` because it's where data quality matters
whereas, in all other cases, unit testing is fine.
I am going, therefore, to proceed with these changes and "pivot"
`netx`. Hopefully, it won't be too painful.
2021-06-23 15:53:12 +02:00
|
|
|
Resolver
|
|
|
|
Logger Logger
|
|
|
|
}
|
|
|
|
|
2021-09-05 14:49:38 +02:00
|
|
|
var _ Resolver = &resolverLogger{}
|
refactor: start pivoting netx (#396)
What do I mean by pivoting? Netx is currently organized by row:
```
| dialer | quicdialer | resolver | ...
saving | | | | ...
errorwrapping | | | | ...
logging | | | | ...
mocking/sys | | | | ...
```
Every row needs to implement saving, errorwrapping, logging, mocking (or
adapting to the system or to some underlying library).
This causes cross package dependencies and, in turn, complexity. For
example, we need the `trace` package for supporting saving.
And `dialer`, `quickdialer`, et al. need to depend on such a package.
The same goes for errorwrapping.
This arrangement further complicates testing. For example, I am
currently working on https://github.com/ooni/probe/issues/1505 and
I realize it need to repeat integration tests in multiple places.
Let's say instead we pivot the above matrix as follows:
```
| saving | errorwrapping | logging | ...
dialer | | | | ...
quicdialer | | | | ...
logging | | | | ...
mocking/sys | | | | ...
...
```
In this way, now every row contains everything related to a specific
action to perform. We can now share code without relying on extra
support packages. What's more, we can write tests and, judding from
the way in which things are made, it seems we only need integration
testing in `errorwrapping` because it's where data quality matters
whereas, in all other cases, unit testing is fine.
I am going, therefore, to proceed with these changes and "pivot"
`netx`. Hopefully, it won't be too painful.
2021-06-23 15:53:12 +02:00
|
|
|
|
2021-09-05 14:49:38 +02:00
|
|
|
func (r *resolverLogger) LookupHost(ctx context.Context, hostname string) ([]string, error) {
|
2021-09-27 16:48:46 +02:00
|
|
|
prefix := fmt.Sprintf("resolve[A,AAAA] %s with %s (%s)", hostname, r.Network(), r.Address())
|
|
|
|
r.Logger.Debugf("%s...", prefix)
|
refactor: start pivoting netx (#396)
What do I mean by pivoting? Netx is currently organized by row:
```
| dialer | quicdialer | resolver | ...
saving | | | | ...
errorwrapping | | | | ...
logging | | | | ...
mocking/sys | | | | ...
```
Every row needs to implement saving, errorwrapping, logging, mocking (or
adapting to the system or to some underlying library).
This causes cross package dependencies and, in turn, complexity. For
example, we need the `trace` package for supporting saving.
And `dialer`, `quickdialer`, et al. need to depend on such a package.
The same goes for errorwrapping.
This arrangement further complicates testing. For example, I am
currently working on https://github.com/ooni/probe/issues/1505 and
I realize it need to repeat integration tests in multiple places.
Let's say instead we pivot the above matrix as follows:
```
| saving | errorwrapping | logging | ...
dialer | | | | ...
quicdialer | | | | ...
logging | | | | ...
mocking/sys | | | | ...
...
```
In this way, now every row contains everything related to a specific
action to perform. We can now share code without relying on extra
support packages. What's more, we can write tests and, judding from
the way in which things are made, it seems we only need integration
testing in `errorwrapping` because it's where data quality matters
whereas, in all other cases, unit testing is fine.
I am going, therefore, to proceed with these changes and "pivot"
`netx`. Hopefully, it won't be too painful.
2021-06-23 15:53:12 +02:00
|
|
|
start := time.Now()
|
|
|
|
addrs, err := r.Resolver.LookupHost(ctx, hostname)
|
2021-06-23 17:00:44 +02:00
|
|
|
elapsed := time.Since(start)
|
|
|
|
if err != nil {
|
2021-09-27 16:48:46 +02:00
|
|
|
r.Logger.Debugf("%s... %s in %s", prefix, err, elapsed)
|
2021-06-23 17:00:44 +02:00
|
|
|
return nil, err
|
|
|
|
}
|
2021-09-27 16:48:46 +02:00
|
|
|
r.Logger.Debugf("%s... %+v in %s", prefix, addrs, elapsed)
|
2021-06-23 17:00:44 +02:00
|
|
|
return addrs, nil
|
refactor: start pivoting netx (#396)
What do I mean by pivoting? Netx is currently organized by row:
```
| dialer | quicdialer | resolver | ...
saving | | | | ...
errorwrapping | | | | ...
logging | | | | ...
mocking/sys | | | | ...
```
Every row needs to implement saving, errorwrapping, logging, mocking (or
adapting to the system or to some underlying library).
This causes cross package dependencies and, in turn, complexity. For
example, we need the `trace` package for supporting saving.
And `dialer`, `quickdialer`, et al. need to depend on such a package.
The same goes for errorwrapping.
This arrangement further complicates testing. For example, I am
currently working on https://github.com/ooni/probe/issues/1505 and
I realize it need to repeat integration tests in multiple places.
Let's say instead we pivot the above matrix as follows:
```
| saving | errorwrapping | logging | ...
dialer | | | | ...
quicdialer | | | | ...
logging | | | | ...
mocking/sys | | | | ...
...
```
In this way, now every row contains everything related to a specific
action to perform. We can now share code without relying on extra
support packages. What's more, we can write tests and, judding from
the way in which things are made, it seems we only need integration
testing in `errorwrapping` because it's where data quality matters
whereas, in all other cases, unit testing is fine.
I am going, therefore, to proceed with these changes and "pivot"
`netx`. Hopefully, it won't be too painful.
2021-06-23 15:53:12 +02:00
|
|
|
}
|
|
|
|
|
2021-09-27 23:09:41 +02:00
|
|
|
func (r *resolverLogger) LookupHTTPS(
|
|
|
|
ctx context.Context, domain string) (*HTTPSSvc, error) {
|
|
|
|
prefix := fmt.Sprintf("resolve[HTTPS] %s with %s (%s)", domain, r.Network(), r.Address())
|
|
|
|
r.Logger.Debugf("%s...", prefix)
|
|
|
|
start := time.Now()
|
|
|
|
https, err := r.Resolver.LookupHTTPS(ctx, domain)
|
|
|
|
elapsed := time.Since(start)
|
|
|
|
if err != nil {
|
|
|
|
r.Logger.Debugf("%s... %s in %s", prefix, err, elapsed)
|
|
|
|
return nil, err
|
|
|
|
}
|
|
|
|
alpn := https.ALPN
|
|
|
|
a := https.IPv4
|
|
|
|
aaaa := https.IPv6
|
|
|
|
r.Logger.Debugf("%s... %+v %+v %+v in %s", prefix, alpn, a, aaaa, elapsed)
|
|
|
|
return https, nil
|
|
|
|
}
|
|
|
|
|
2021-09-05 14:49:38 +02:00
|
|
|
// resolverIDNA supports resolving Internationalized Domain Names.
|
2021-08-17 11:02:12 +02:00
|
|
|
//
|
2021-08-17 10:29:06 +02:00
|
|
|
// See RFC3492 for more information.
|
2021-09-05 14:49:38 +02:00
|
|
|
type resolverIDNA struct {
|
2021-08-17 10:29:06 +02:00
|
|
|
Resolver
|
|
|
|
}
|
|
|
|
|
2021-09-05 14:49:38 +02:00
|
|
|
func (r *resolverIDNA) LookupHost(ctx context.Context, hostname string) ([]string, error) {
|
2021-08-17 10:29:06 +02:00
|
|
|
host, err := idna.ToASCII(hostname)
|
|
|
|
if err != nil {
|
|
|
|
return nil, err
|
|
|
|
}
|
|
|
|
return r.Resolver.LookupHost(ctx, host)
|
|
|
|
}
|
2021-09-05 20:12:05 +02:00
|
|
|
|
2021-09-27 23:09:41 +02:00
|
|
|
func (r *resolverIDNA) LookupHTTPS(
|
|
|
|
ctx context.Context, domain string) (*HTTPSSvc, error) {
|
|
|
|
host, err := idna.ToASCII(domain)
|
|
|
|
if err != nil {
|
|
|
|
return nil, err
|
|
|
|
}
|
|
|
|
return r.Resolver.LookupHTTPS(ctx, host)
|
|
|
|
}
|
|
|
|
|
2021-09-05 20:12:05 +02:00
|
|
|
// resolverShortCircuitIPAddr recognizes when the input hostname is an
|
|
|
|
// IP address and returns it immediately to the caller.
|
|
|
|
type resolverShortCircuitIPAddr struct {
|
|
|
|
Resolver
|
|
|
|
}
|
|
|
|
|
|
|
|
func (r *resolverShortCircuitIPAddr) LookupHost(ctx context.Context, hostname string) ([]string, error) {
|
|
|
|
if net.ParseIP(hostname) != nil {
|
|
|
|
return []string{hostname}, nil
|
|
|
|
}
|
|
|
|
return r.Resolver.LookupHost(ctx, hostname)
|
|
|
|
}
|
2021-09-05 20:41:46 +02:00
|
|
|
|
2021-09-29 20:21:25 +02:00
|
|
|
// ErrNoResolver is the type of error returned by "without resolver"
|
|
|
|
// dialer when asked to dial for and endpoint containing a domain name,
|
|
|
|
// since they can only dial for endpoints containing IP addresses.
|
2021-09-05 20:41:46 +02:00
|
|
|
var ErrNoResolver = errors.New("no configured resolver")
|
|
|
|
|
|
|
|
// nullResolver is a resolver that is not capable of resolving
|
|
|
|
// domain names to IP addresses and always returns ErrNoResolver.
|
|
|
|
type nullResolver struct{}
|
|
|
|
|
|
|
|
func (r *nullResolver) LookupHost(ctx context.Context, hostname string) (addrs []string, err error) {
|
|
|
|
return nil, ErrNoResolver
|
|
|
|
}
|
|
|
|
|
|
|
|
func (r *nullResolver) Network() string {
|
|
|
|
return "null"
|
|
|
|
}
|
|
|
|
|
|
|
|
func (r *nullResolver) Address() string {
|
|
|
|
return ""
|
|
|
|
}
|
|
|
|
|
|
|
|
func (r *nullResolver) CloseIdleConnections() {
|
2021-09-08 22:48:10 +02:00
|
|
|
// nothing to do
|
2021-09-05 20:41:46 +02:00
|
|
|
}
|
2021-09-07 19:56:42 +02:00
|
|
|
|
2021-09-27 23:09:41 +02:00
|
|
|
func (r *nullResolver) LookupHTTPS(
|
|
|
|
ctx context.Context, domain string) (*HTTPSSvc, error) {
|
|
|
|
return nil, ErrNoResolver
|
|
|
|
}
|
|
|
|
|
2021-09-07 19:56:42 +02:00
|
|
|
// resolverErrWrapper is a Resolver that knows about wrapping errors.
|
|
|
|
type resolverErrWrapper struct {
|
|
|
|
Resolver
|
|
|
|
}
|
|
|
|
|
|
|
|
var _ Resolver = &resolverErrWrapper{}
|
|
|
|
|
|
|
|
func (r *resolverErrWrapper) LookupHost(ctx context.Context, hostname string) ([]string, error) {
|
|
|
|
addrs, err := r.Resolver.LookupHost(ctx, hostname)
|
|
|
|
if err != nil {
|
2021-09-28 12:42:01 +02:00
|
|
|
return nil, NewErrWrapper(ClassifyResolverError, ResolveOperation, err)
|
2021-09-07 19:56:42 +02:00
|
|
|
}
|
|
|
|
return addrs, nil
|
|
|
|
}
|
2021-09-27 23:09:41 +02:00
|
|
|
|
|
|
|
func (r *resolverErrWrapper) LookupHTTPS(
|
|
|
|
ctx context.Context, domain string) (*HTTPSSvc, error) {
|
|
|
|
out, err := r.Resolver.LookupHTTPS(ctx, domain)
|
|
|
|
if err != nil {
|
2021-09-28 12:42:01 +02:00
|
|
|
return nil, NewErrWrapper(ClassifyResolverError, ResolveOperation, err)
|
2021-09-27 23:09:41 +02:00
|
|
|
}
|
|
|
|
return out, nil
|
|
|
|
}
|