ooni-probe-cli/pkg/oonimkall/eventemitter.go

48 lines
1.3 KiB
Go
Raw Normal View History

package oonimkall
// eventEmitter emits event on a channel
type eventEmitter struct {
disabled map[string]bool
fix(oonimkall): improve channel management pattern (#621) 1. add eof channel to event emitter and use this channel as signal that we shouldn't be sending anymore instead of using a pattern where we use a timer to decide sending has timed out (because we're using a buffered channel, it is still possible for some evetns to end up in the channel if there is space, but this is not a concern, because the events will be deleted when the channel itself is gone); 2. refactor all tests where we assumed the output channel was closed to actually use a parallel "eof" channel and use it as signal we should not be sending anymore (not strictly required but still the right thing to do in terms of using consistent patterns); 3. modify how we construct a runner so that it passes to the event emitter an "eof" channel and closes this channel when the main goroutine running the task is terminating; 4. modify the task to signal events such as "task goroutine started" and "task goroutine stopped" using channels, which helps to write much more correct tests; 5. take advantage of the previous change to improve the test that ensures we're not blocking for a small number of events and also improve the name of such a test to reflect what it's testing. The related issue in term of fixing the channel usage pattern is https://github.com/ooni/probe/issues/1438. Regarding improving testability, instead, the correct reference issue is https://github.com/ooni/probe/issues/1903. There are possibly more changes to apply here to improve this package and its testability, but let's land this diff first and then see how much time is left for further improvements. I've run unit and integration tests with `-race` locally. This diff will need to be backported to `release/3.11`.
2021-12-01 15:40:25 +01:00
eof <-chan interface{}
out chan<- *event
}
// newEventEmitter creates a new Emitter
fix(oonimkall): improve channel management pattern (#621) 1. add eof channel to event emitter and use this channel as signal that we shouldn't be sending anymore instead of using a pattern where we use a timer to decide sending has timed out (because we're using a buffered channel, it is still possible for some evetns to end up in the channel if there is space, but this is not a concern, because the events will be deleted when the channel itself is gone); 2. refactor all tests where we assumed the output channel was closed to actually use a parallel "eof" channel and use it as signal we should not be sending anymore (not strictly required but still the right thing to do in terms of using consistent patterns); 3. modify how we construct a runner so that it passes to the event emitter an "eof" channel and closes this channel when the main goroutine running the task is terminating; 4. modify the task to signal events such as "task goroutine started" and "task goroutine stopped" using channels, which helps to write much more correct tests; 5. take advantage of the previous change to improve the test that ensures we're not blocking for a small number of events and also improve the name of such a test to reflect what it's testing. The related issue in term of fixing the channel usage pattern is https://github.com/ooni/probe/issues/1438. Regarding improving testability, instead, the correct reference issue is https://github.com/ooni/probe/issues/1903. There are possibly more changes to apply here to improve this package and its testability, but let's land this diff first and then see how much time is left for further improvements. I've run unit and integration tests with `-race` locally. This diff will need to be backported to `release/3.11`.
2021-12-01 15:40:25 +01:00
func newEventEmitter(disabledEvents []string, out chan<- *event,
eof <-chan interface{}) *eventEmitter {
ee := &eventEmitter{eof: eof, out: out}
ee.disabled = make(map[string]bool)
for _, eventname := range disabledEvents {
ee.disabled[eventname] = true
}
return ee
}
// EmitFailureStartup emits the failureStartup event
func (ee *eventEmitter) EmitFailureStartup(failure string) {
ee.EmitFailureGeneric(failureStartup, failure)
}
// EmitFailureGeneric emits a failure event
func (ee *eventEmitter) EmitFailureGeneric(name, failure string) {
ee.Emit(name, eventFailure{Failure: failure})
}
// EmitStatusProgress emits the status.Progress event
func (ee *eventEmitter) EmitStatusProgress(percentage float64, message string) {
ee.Emit(statusProgress, eventStatusProgress{Message: message, Percentage: percentage})
}
// Emit emits the specified event
func (ee *eventEmitter) Emit(key string, value interface{}) {
if ee.disabled[key] {
return
}
fix(oonimkall): improve channel management pattern (#621) 1. add eof channel to event emitter and use this channel as signal that we shouldn't be sending anymore instead of using a pattern where we use a timer to decide sending has timed out (because we're using a buffered channel, it is still possible for some evetns to end up in the channel if there is space, but this is not a concern, because the events will be deleted when the channel itself is gone); 2. refactor all tests where we assumed the output channel was closed to actually use a parallel "eof" channel and use it as signal we should not be sending anymore (not strictly required but still the right thing to do in terms of using consistent patterns); 3. modify how we construct a runner so that it passes to the event emitter an "eof" channel and closes this channel when the main goroutine running the task is terminating; 4. modify the task to signal events such as "task goroutine started" and "task goroutine stopped" using channels, which helps to write much more correct tests; 5. take advantage of the previous change to improve the test that ensures we're not blocking for a small number of events and also improve the name of such a test to reflect what it's testing. The related issue in term of fixing the channel usage pattern is https://github.com/ooni/probe/issues/1438. Regarding improving testability, instead, the correct reference issue is https://github.com/ooni/probe/issues/1903. There are possibly more changes to apply here to improve this package and its testability, but let's land this diff first and then see how much time is left for further improvements. I've run unit and integration tests with `-race` locally. This diff will need to be backported to `release/3.11`.
2021-12-01 15:40:25 +01:00
// Prevent this goroutine from blocking on `ee.out` if the caller
// has already told us it's not going to accept more events.
[forwardport] fix(oonimkall): don't close channel to signal end of task (#619) (#620) This diff forward ports 90bf0629b957c912a5a6f3bb6c98ad3abb5a2ff6 to `master`. If we close the channel to signal the end of a task we may panic when some background goroutine tries to post on the channel. This bug is rare but may happen. See for example https://github.com/ooni/probe/issues/1438. How can we improve? First, let us add a timeout when sending to the channel. Given that the channel is buffered and we have a generous timeout (1/4 of a second), it's unlikely we will really block. But, in the event in which a late message appears, we'll eventually _unblock_ when sending with a timeout. So, now we don't have to worry anymore about leaking forever a goroutine. Then, let us change the protocol with which we signal that a task is done. We used to close the channel. Now, instead we just synchronously post a nil on the channel when done. In turn, we interpret this nil to mean that the task is done when we receive messages. The _main_ different with respect to before is that now we are asking the consumer of our API to drain the channel. Because before we had a blocking channel, it seems to me we were already requiring the consumer of the API to do that. Which means, I think in practical terms it did not change much. Finally, acknowledge that we don't need a specific state variable to tell us we're done and simplify a little bit the API by just making isRunning private and using the "we're done" signal to determine whether we've stopped running the task. All these changes should be enough to close https://github.com/ooni/probe/issues/1438.
2021-11-26 22:18:45 +01:00
select {
case ee.out <- &event{Key: key, Value: value}:
fix(oonimkall): improve channel management pattern (#621) 1. add eof channel to event emitter and use this channel as signal that we shouldn't be sending anymore instead of using a pattern where we use a timer to decide sending has timed out (because we're using a buffered channel, it is still possible for some evetns to end up in the channel if there is space, but this is not a concern, because the events will be deleted when the channel itself is gone); 2. refactor all tests where we assumed the output channel was closed to actually use a parallel "eof" channel and use it as signal we should not be sending anymore (not strictly required but still the right thing to do in terms of using consistent patterns); 3. modify how we construct a runner so that it passes to the event emitter an "eof" channel and closes this channel when the main goroutine running the task is terminating; 4. modify the task to signal events such as "task goroutine started" and "task goroutine stopped" using channels, which helps to write much more correct tests; 5. take advantage of the previous change to improve the test that ensures we're not blocking for a small number of events and also improve the name of such a test to reflect what it's testing. The related issue in term of fixing the channel usage pattern is https://github.com/ooni/probe/issues/1438. Regarding improving testability, instead, the correct reference issue is https://github.com/ooni/probe/issues/1903. There are possibly more changes to apply here to improve this package and its testability, but let's land this diff first and then see how much time is left for further improvements. I've run unit and integration tests with `-race` locally. This diff will need to be backported to `release/3.11`.
2021-12-01 15:40:25 +01:00
case <-ee.eof:
[forwardport] fix(oonimkall): don't close channel to signal end of task (#619) (#620) This diff forward ports 90bf0629b957c912a5a6f3bb6c98ad3abb5a2ff6 to `master`. If we close the channel to signal the end of a task we may panic when some background goroutine tries to post on the channel. This bug is rare but may happen. See for example https://github.com/ooni/probe/issues/1438. How can we improve? First, let us add a timeout when sending to the channel. Given that the channel is buffered and we have a generous timeout (1/4 of a second), it's unlikely we will really block. But, in the event in which a late message appears, we'll eventually _unblock_ when sending with a timeout. So, now we don't have to worry anymore about leaking forever a goroutine. Then, let us change the protocol with which we signal that a task is done. We used to close the channel. Now, instead we just synchronously post a nil on the channel when done. In turn, we interpret this nil to mean that the task is done when we receive messages. The _main_ different with respect to before is that now we are asking the consumer of our API to drain the channel. Because before we had a blocking channel, it seems to me we were already requiring the consumer of the API to do that. Which means, I think in practical terms it did not change much. Finally, acknowledge that we don't need a specific state variable to tell us we're done and simplify a little bit the API by just making isRunning private and using the "we're done" signal to determine whether we've stopped running the task. All these changes should be enough to close https://github.com/ooni/probe/issues/1438.
2021-11-26 22:18:45 +01:00
}
}