Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

IllegalReferenceCountException with HTTP/2 if maximum number of streams is exceeded

See original GitHub issue

Expected behavior

No exception should be thrown.

Actual behavior

Http2ConnectionHandler.exceptionCaught is called with an IllegalReferenceCountException


io.netty.util.IllegalReferenceCountException: refCnt: 0, increment: 1
	at io.netty.buffer.AbstractReferenceCountedByteBuf.release0(AbstractReferenceCountedByteBuf.java:100)
	at io.netty.buffer.AbstractReferenceCountedByteBuf.release(AbstractReferenceCountedByteBuf.java:84)
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:273)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
	at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1379)
	at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1158)
	at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1193)
	at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:489)
	at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:428)
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:265)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1359)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:935)
	at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:801)
	at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:404)
	at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:304)
	at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
	at java.lang.Thread.run(Thread.java:748)

Steps to reproduce

Call this 200 times in an Http2ConnectionHandler

        ChannelPromise promise = ctx.newPromise();
        int outStreamId = connection().local().incrementAndGetNextStreamId();
   encoder().writeHeaders(ctx, outStreamId, http2Headers, 0, true, promise);
        flush(ctx);

Netty version

Netty 4.1.17.Final Tcnative 2.0.7.Final Using Epoll Transport and OpenSSL

JVM version (e.g. `java -version`)

Java™ SE Runtime Environment (build 1.8.0_144-b01) Java HotSpot™ 64-Bit Server VM (build 25.144-b01, mixed mode)

OS version (e.g. `uname -a`)

Linux 4.9.44+grsec

Issue Analytics

State:
Created 6 years ago
Comments:9 (9 by maintainers)

Top GitHub Comments

1reaction

eun-icecommented, Dec 2, 2017

The problem arises without (actively) reading any data and seems to happen on shutdown. In practice we send push notifications to google, if we send to many at once the above exception happens after netty hits its maximum open streams limit. We solved it for our use case to limit the number of streams we open, but the exception above hints at an underlying problem. Let me check if I can provide a stand-alone reproducer soon. It will take me s few days though

0reactions

eun-icecommented, Dec 22, 2017

Sorry for bothering you guys and thanks for looking into it.

Top Results From Across the Web

configure spring.codec.max-in-memory-size When using ...

It is hardcoded but overridden by the config. "Limit on the number of bytes that can be buffered whenever the input stream needs...

AbstractHttp2ConnectionHandler...

Returns if the encoder should queue frames if the maximum number of concurrent streams would otherwise be exceeded. encoderEnforceMaxConcurrentStreams.

Index (Netty API Reference (4.1.59.Final)) - javadoc.io

Constructor using NO LIMIT and default max time as delay allowed value of ... if the maximum number of concurrent streams would otherwise...

BIG-IP system occasionally actively close HTTP2 connection

If you find Max Streams Exceeded is keeping increase, that indicates a higher value of Concurrent Streams Per Connection in http/2 profile ...

Mint.HTTP2 — Mint v1.4.2 - HexDocs

When a connection is closed for writing, the client cannot send requests or ... Tells what is the maximum number of streams that...