OutOfDirectMemoryError
See original GitHub issueDescribe the bug We’re using a proxy to connect to our pulsar cluster and the proxy was failing to send messages to our broker. The error on the broker is:
WARN io.netty.channel.ChannelInitializer - Failed to initialize a channel. Closing: [id: .... ]
io.netty.util.internal.OutOfDirectMemoryError: failed to allocated 866 byte(s) of direct memory (used: 4Gb, max: 4Gb)
at io.netty.util.internal.PlatformDependent.incrementMemoryCounter(PlatformDependent.java:640)
at io.netty.util.internal.PlatformDependent.allocatedDirectNoCleaner(PlatformDependent.java:594)
...
Expected behavior I’d expect the proxy to send data to another broker, but I also don’t see why this broker would have an OOM when it should be persisting to disk for the most part?
Desktop (please complete the following information): CentOS 7.6 Pulsar 2.2.1
Additional context
-Xms4g -Xmx16g -XX:MaxDirectMemorySize=4g
Issue Analytics
- State:
- Created 5 years ago
- Comments:5 (3 by maintainers)
Top Results From Across the Web
How to find a root cause of the following netty error: io.netty.util ...
OutOfDirectMemoryError : failed to allocate 16777216 byte(s) of direct memory (used: 117440512, max: 119537664) at io.netty.util.internal.
Read more >Netty OutOfDirectMemoryError: failed to allocate direct memory
We have a direct memory allocated of about 14gb. We have 10gb dedicated to a cache (separated from reactor-netty). We expected that for...
Read more >OutOfDirectMemoryError (Netty/Common 4.1.43.Final API)
OutOfMemoryError that is throws if PlatformDependent.allocateDirectNoCleaner(int) can not allocate a new ByteBuffer due memory restrictions. See Also: ...
Read more >OutOfDirectMemoryError - Google Groups
OutOfDirectMemoryError : failed to allocate 16777216 byte(s) of direct memory (used: 1056964615, max: 1073741824). I looked at the history of this user group ......
Read more >OutOfDirectMemoryError (Netty API Reference (4.1.85.Final))
OutOfMemoryError that is throws if PlatformDependent.allocateDirectNoCleaner(int) can not allocate a new ByteBuffer due memory restrictions.
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
@bputt This would be kind of addressed in 2.3. The changes were described in https://github.com/apache/pulsar/wiki/PIP-24%3A-Simplify-memory-settings .
The implementation was actually done in BookKeeper and will be available in 4.9.0 release (which is currently being voted). Once that’s available, we’ll add the hooks in Pulsar broker as well to ensure that under any circumstances, the broker will restart when a OOM exception occurs. (at the same mitigations for that not to happen as easily are also there).
@merlimat do you have an expected release date for 2.3?