Lz4FrameEncoder (and other compressors) Acknowledge Writes Too Early

Expected behaviour: The promise for a write should be completed only once the write makes it to the network

Actual behaviour: Buffering encoders that extend MessageToByteEncoder, most notably Lz4FrameEncoder and other compressors, will notify the promise before the frame containing the message reaches the network.

This behaviour is triggered by MessageToByteEncoder.write which is designed for 1:1 conversions from message to ByteBuf. For the affected compressors, the encode method returns a ByteBuf that does not fully contain the message, unless it happens to precisely fill the buffer.

Either an empty ByteBuf is returned, or one containing only part of the message, with the remainder in both cases left in the buffer. In either case, the supplied promise will be attached to the write of this ByteBuf (empty or otherwise), and not to the ByteBuf that contains the end of the message.

Netty version: 4.0+ (I think, certainly confirmed on 4.1)

[belliottsmith]

This is definitely a bug - sorry if I should have labelled it as such at filing (not sure I have the rights to do so though)

Agreed that we should extend MessageToByteEncoder

A more “correct” fix would be to standardise the buffering encoders on a class that can handle the buffering for them. But something like what you propose is certainly easier and less invasive. I’ll take a look at rustling up a candidate soon.

[belliottsmith]

Sorry @normanmaurer, I did not and since forgot. It turns out it’s harder to keep up with your regular responsibilities when you’re distracted by a cute baby in your personal time.

I’d still be willing to take a look, but you might have to bear with me another few weeks. Please don’t feel like you’re treading on my toes if you or somebody else wants to address it though.