Testing networking services is hard

What is wrong?

Testing networking services in isolation is quite difficult right now, as discussed in #688 .

How can it be fixed

I think we should try to separate different layers of the code more so that we can replace them easily. Here’s a concrete proposal:

• Each BasePeer gets a BasePeerInterface instance. BasePeer calls BasePeerInterface.send_command(command), BasePeerInterface calls BasePeer.process_command(command) (alternatively, both classes have a command_queue on which they put commands on).
• To allow this, Commands keep parameters as instance variables.
• StreamPeerInterface (or just PeerInterface) writes commands to the output stream and reads them from the input stream, just like the current BasePeer does.
• For tests, there’s a MockPeerInterface that passes commands through to a shared Network object that sends them to the receiving MockPeerInterface. That means that nothing goes over a network interface and nothing needs to be encoded or decoded.
• Add a MockDiscovery which gets a list of peer pools to “discover”.
• PeerPool makes the handshake function configurable, either by taking it as a constructor argument or by making a BasePeerPool and requiring it to implement it in sub classes. We add a mock_handshake that looks up the remote in Network and creates a new peer with a MockPeerInterface.

I haven’t thought this through completely (especially how it affects PeerPool and Server), but don’t have more time just now and wanted to write it down. Will update this tonight when I’ll hopefully had some more time.

[gsalgado]

One problem with those big architectural redesigns is that usually nobody will have the time/willpower to implement them, so we end up doing things in a suboptimal way waiting for a redesign that will never come. Instead I very much prefer to make small improvements every time I touch a piece of code. For instance, although testing async code is hard and we’re severely lacking on infrastructure for that, we do have some and with some small improvements the existing sharding tests can be a lot simpler and much less fragile

[pipermerriam]

I think that’s a fair point, and probably a good conservative approach. I’d be game to re-visit this when/if someone feels like they have a sufficiently good handle on how to do it and the time to execute on it.