Leak sending huge amount of x-protobuf (byte[]) in a request to an http controller

Expected behavior

When sending http requests to a Controller with huge data, no memory leak should happen.

Actual behavior

When sending a huge amount of data as application/x-protobuf to a @Controller a memory leak happens and a log trace appears.

Steps to reproduce

All of this has been tested in Linux (Ubuntu)

• Clone this project https://github.com/dcalap/micronaut-netty-leak-problem build it and run it in Docker
• Install Apache Benchmark to generate a big amount of requests
• Generate a “big” file to be sent: dd if=/dev/zero of=post-500mb.data count=1024 bs=512000
• Send it to our controller through Apache Benchmark: ab -T 'application/x-protobuf' -n 9999 -c 10 -p post-500mb.data http://[HOST_IP]:8080/dummy/issue

Minimal yet complete reproducer code (or URL to code)

https://github.com/dcalap/micronaut-netty-leak-problem

Netty version

4.1.38 (the one included in micronaut 1.2.2)

JVM version (e.g. java -version)

adoptopenjdk/openjdk11-openj9:jdk-11.0.1.13-alpine-slim

Log

10:20:11.351 [nioEventLoopGroup-1-8] ERROR io.netty.util.ResourceLeakDetector - LEAK: ByteBuf.release() was not called before it's garbage-collected. See https://netty.io/wiki/reference-counted-objects.html for more information. Recent access records: Created at: io.netty.buffer.PooledByteBufAllocator.newDirectBuffer(PooledByteBufAllocator.java:349) io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:187) io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:178) io.netty.buffer.AbstractByteBufAllocator.ioBuffer(AbstractByteBufAllocator.java:139) io.netty.channel.DefaultMaxMessagesRecvByteBufAllocator$MaxMessageHandle.allocate(DefaultMaxMessagesRecvByteBufAllocator.java:114) io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:147) io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:697) io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:632) io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:549) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:511) io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918) io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base/java.lang.Thread.run(Thread.java:825)

[normanmaurer]

@dcalap @marcosflobo the log is truncated (as noted at the end). Please use a higher “io.netty.leakDetection.targetRecords” setting and report back. Something like -Dio.netty.leakDetection.targetRecords=32.

[dcalap]

Because it looks like we read something in the buffer and then dispatched it to the pipeline.

Micronaut guys addressed us to netty. If you can check the referenced issue to see if maybe is netty stuff or you can talk to each other it would be great 😃 Thanks!