question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Memory leak when creating ReferenceCountedSslServerContext instances

See original GitHub issue

We’re currently running a 4.1.7.Final-SNAPSHOT version of Netty that pre-dates the official 4.1.7.Final release (our SNAPSHOT is roughly from January 6, 2017).

We today attempted a release with 4.1.8.Final-SNAPSHOT and I’ve tried 4.1.7.Final as well. There is suddenly a large number of the following (new) Exceptions and the server runs out of memory quickly until Linux’s oom_killer steps in and axes the process. There are no Java OOMEs nor do YourKit memory snapshots show anything out of the ordinary which points towards direct memory being leaked.

Looking the recent commit history there appears to be this dd055c01c78c3a6ce5b9195890486a6e1ab072d2 change that is potentially related (happened after January 6 but before 4.1.7.Final release).

javax.net.ssl.SSLException: error:140D00CF:SSL routines:SSL_write:protocol is shutdown
	at io.netty.handler.ssl.ReferenceCountedOpenSslEngine.shutdownWithError(ReferenceCountedOpenSslEngine.java:719) ~[netty-all-4.1.7.Final.jar:4.1.7.Final]
	at io.netty.handler.ssl.ReferenceCountedOpenSslEngine.shutdownWithError(ReferenceCountedOpenSslEngine.java:708) ~[netty-all-4.1.7.Final.jar:4.1.7.Final]
	at io.netty.handler.ssl.ReferenceCountedOpenSslEngine.wrap(ReferenceCountedOpenSslEngine.java:685) ~[netty-all-4.1.7.Final.jar:4.1.7.Final]
	at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:509) ~[?:1.8.0_102]
	at io.netty.handler.ssl.SslHandler.wrap(SslHandler.java:746) ~[netty-all-4.1.7.Final.jar:4.1.7.Final]
	at io.netty.handler.ssl.SslHandler.wrap(SslHandler.java:578) ~[netty-all-4.1.7.Final.jar:4.1.7.Final]
	at io.netty.handler.ssl.SslHandler.wrapAndFlush(SslHandler.java:550) ~[netty-all-4.1.7.Final.jar:4.1.7.Final]
	at io.netty.handler.ssl.SslHandler.flush(SslHandler.java:531) ~[netty-all-4.1.7.Final.jar:4.1.7.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:777) [netty-all-4.1.7.Final.jar:4.1.7.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:769) [netty-all-4.1.7.Final.jar:4.1.7.Final]
	at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:750) [netty-all-4.1.7.Final.jar:4.1.7.Final]
	at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.flush(CombinedChannelDuplexHandler.java:530) [netty-all-4.1.7.Final.jar:4.1.7.Final]
	at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115) [netty-all-4.1.7.Final.jar:4.1.7.Final]
	at io.netty.channel.CombinedChannelDuplexHandler.flush(CombinedChannelDuplexHandler.java:355) [netty-all-4.1.7.Final.jar:4.1.7.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:777) [netty-all-4.1.7.Final.jar:4.1.7.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:769) [netty-all-4.1.7.Final.jar:4.1.7.Final]
	at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:750) [netty-all-4.1.7.Final.jar:4.1.7.Final]
	at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final.jar:4.1.7.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:777) [netty-all-4.1.7.Final.jar:4.1.7.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:769) [netty-all-4.1.7.Final.jar:4.1.7.Final]
	at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:750) [netty-all-4.1.7.Final.jar:4.1.7.Final]
	at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final.jar:4.1.7.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:777) [netty-all-4.1.7.Final.jar:4.1.7.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:769) [netty-all-4.1.7.Final.jar:4.1.7.Final]
	at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:750) [netty-all-4.1.7.Final.jar:4.1.7.Final]
	at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115) [netty-all-4.1.7.Final.jar:4.1.7.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:777) [netty-all-4.1.7.Final.jar:4.1.7.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:769) [netty-all-4.1.7.Final.jar:4.1.7.Final]
	at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:750) [netty-all-4.1.7.Final.jar:4.1.7.Final]
	at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115) [netty-all-4.1.7.Final.jar:4.1.7.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:777) [netty-all-4.1.7.Final.jar:4.1.7.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:769) [netty-all-4.1.7.Final.jar:4.1.7.Final]
	at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:750) [netty-all-4.1.7.Final.jar:4.1.7.Final]
	at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115) [netty-all-4.1.7.Final.jar:4.1.7.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:777) [netty-all-4.1.7.Final.jar:4.1.7.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:769) [netty-all-4.1.7.Final.jar:4.1.7.Final]
	at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:750) [netty-all-4.1.7.Final.jar:4.1.7.Final]
	at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final.jar:4.1.7.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:777) [netty-all-4.1.7.Final.jar:4.1.7.Final]
...

Issue Analytics

  • State:closed
  • Created 7 years ago
  • Comments:46 (46 by maintainers)

github_iconTop GitHub Comments

1reaction
normanmaurercommented, Jan 20, 2017

@doom369 already its on maven central now… See also: https://github.com/netty/netty/pull/6259

1reaction
normanmaurercommented, Jan 19, 2017

lol… You beat me by a few secs 😉

Read more comments on GitHub >

github_iconTop Results From Across the Web

Memory leak Using OpenSSL APIs - Stack Overflow
You need to set the parameter object, then free it locally. the reference count should retain it in single-ownership within the context (and ......
Read more >
Valgrind reports "still reachable" memory when using SSL #263
valgrind reports "still reachable" memory when enabling SSL. The following command should reproduce the issue.
Read more >
possible memory leak when connecting (fixed for me)
The problem that I found was in several instances the X509_free calls did. NOT free the data (because at the last attempt to...
Read more >
possible memory leak when connecting (fixed for me)' - MARC
[next in thread] List: openssl-users Subject: Re: possible memory leak when connecting ... When you try to > free it, it just reduces...
Read more >
subject:"Memory Leaks" - The Mail Archive
One thing I am doing is creating a single SSL_CTX and using it for multiple SSL sessions. I presume the SSL_CTX is reference...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found