If the request contains a string of millions of lengths, and if the request is sent at a very high rate, will it cause an io.netty.util.internal.OutOfDirectMemoryError exception?

If the request contains a string of millions of lengths, and if the request is sent at a very high rate, will it cause an io.netty.util.internal.OutOfDirectMemoryError exception?

BTW, I have tried many times and have not found the LEAK log…

Expected behavior

Netty service runs steadily without reporting OOM exceptions.

Actual behavior

io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 16777216 byte(s) of direct memory (used: 788529159, max: 804257792)
	at io.netty.util.internal.PlatformDependent.incrementMemoryCounter(PlatformDependent.java:742)
	at io.netty.util.internal.PlatformDependent.allocateDirectNoCleaner(PlatformDependent.java:697)
	at io.netty.buffer.PoolArena$DirectArena.allocateDirect(PoolArena.java:758)
	at io.netty.buffer.PoolArena$DirectArena.newChunk(PoolArena.java:734)
	at io.netty.buffer.PoolArena.allocateNormal(PoolArena.java:245)
	at io.netty.buffer.PoolArena.allocate(PoolArena.java:215)
	at io.netty.buffer.PoolArena.allocate(PoolArena.java:147)
	at io.netty.buffer.PooledByteBufAllocator.newDirectBuffer(PooledByteBufAllocator.java:356)
	at io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:187)
	at io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:178)
	at io.netty.channel.unix.PreferredDirectByteBufAllocator.ioBuffer(PreferredDirectByteBufAllocator.java:53)
	at io.netty.channel.DefaultMaxMessagesRecvByteBufAllocator$MaxMessageHandle.allocate(DefaultMaxMessagesRecvByteBufAllocator.java:114)
	at io.netty.channel.epoll.EpollRecvByteAllocatorHandle.allocate(EpollRecvByteAllocatorHandle.java:75)
	at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:777)
	at io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe$1.run(AbstractEpollChannel.java:387)
	at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
	at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
	at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:387)
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	at java.lang.Thread.run(Thread.java:748)

Steps to reproduce

Many requests containing a string of millions of lengths are sent at a very high rate

Minimal yet complete reproducer code (or URL to code)

Netty version

4.1.47.Final

JVM version (e.g. java -version)

java version "1.8.0_241"
Java(TM) SE Runtime Environment (build 1.8.0_241-b07)
Java HotSpot(TM) 64-Bit Server VM (build 25.241-b07, mixed mode)

OS version (e.g. uname -a)

Linux x86_64 GNU/Linux

[normanmaurer]

Closing this

[asdf2014]

Hi, @normanmaurer . I am so glad to tell you that this problem has been solved by limiting the number of active channels. As mentioned by @johnou, the speed of writing data from Netty is faster than the speed we can process in the back-end program, which cause the direct memory couldn’t be released in time, resulting in OOM exceptions. Thank you both for following up this issue, thank you very much!