Implement async Netty pipeline

Problem

Netty API is solely callback based, and a next ChannelHandler should be added to the pipeline synchronously and immediately right after the handler complete some intermediate task (i.e. inside a ChannelHandler callback when the last event is received). Sometimes the next handler should be added even before the old handler is removed, since upon remove old handler may forward cached but unprocessed bytes toward pipeline and the next handler should already be in place to pick up those bytes. This design causes quite inconvenient UX especially when we trying to deliver another API (libp2p in our case) based on Netty architecture. This limitation can’t be workaround except with some dirty solutions like blocking ChannelHandler callbacks.

What we want

We want to add a next handler asynchronously when a previous one completes its task. For example the following code should be possible:

Future<Connection> connF = transport.dial(addr);
Future<SecureConnection> secConnF = connF
    .thenApply(conn -> conn.secure(new SecurityHandler()));
Future<MplexedConnection> mplexConnF = secConnF
    .thenApply(secConn -> secConn.multiplex(new MultiplexHandler()));
Future<Stream> streamF = mplexConnF
    .thenApply(mplexConn.createStream());
streamF.thenAccept(stream -> stream.getChannel().pipeline().addLast(applicatioHandler))

Possible solution

The main reason is that default Netty pipeline just drops unprocessed events when reaching the end of pipeline. Thus we can implement pipeline which doesn’t drop events but queues them and when a new handler added replays all events to it. To prevent unbound data receiving the pipeline should setup AUTO_READ flag from the point when first unhandled data queued and the point when next handler added

[jezhiggins]

By finding the appropriate place for Netty to sit in our stack, and with the judicious application of new interfaces I believe we can solve, or certainly, ease the problems Anton identifies, make working with libp2p more straightforward, and make the implementation less complex and therefore more robust.

I’m going to continue on the branch linked above, and I’d welcome your comments, suggestions, or alternative proposals. Thanks.

[jezhiggins]

Thanks @shahankhatch. I think you’ve captured what I’m trying to say very well. We should be trying to find the appropriate abstractions at each level, making the common cases as straightforward as we can, without precluding the difficult ones.

I’ll continue exploring on the refactor/push-down-netty branch, with a view to raising a PR against the develop branch when that’s settled down. Might help us to have actual examples to discuss.

Equally, I’m open to alternative approaches to explore or particular places in the code it might be worth paying particular attention to.