[BUG] Cosmos SDK Netty ByteBuf leak
See original GitHub issueDescribe the bug There is a leak in the Cosmos SDK when using the DIRECT connection mode.
The problem seems to only become obvious when services are under continuous medium load. We’ve had Cosmos deployed for many months and this hasn’t been an issue, but shortly after going live and having sustained traffic increase above a certain level, all our services started running out of heap memory and experiencing OutOfMemory exceptions.
A leak this bad is highly concerning. Fortunately we were able to switch our service to use GATEWAY mode to bypass the issue.
We went through several mitigation steps, including:
- increasing the heap size
- pulling heap dumps to analyze leaks
- scaling our deployments
- downgrading reactor netty from 0.9.4 to 0.9.0
- Upgrade Cosmos SDK from 3.5.0 to 3.7.0
All to have it become abundantly clear that the leak was coming from the Cosmos SDK, and that there was nothing we could do.
We subsequently enabled ADVANCED Netty leak detection to get a record of the last code to access the ByteBuf. That log showed that the last access is a call to release()
at RntbdResponse.java:198
. That code has suspicious comments, but more importantly it helped me confirm that attempting to switch to GATEWAY connection mode would likely bypass the leaking behavior, which it did.
Netty Leak Detection Log
--2020-03-10 15:18:36.798 -ERROR -- [rntbd-nio-2-1] io.netty.util.ResourceLeakDetector : LEAK: ByteBuf.release() was not called before it's garbage-collected. See https://netty.io/wiki/reference-counted-objects.html for more information.
Recent access records:
#1:
com.azure.data.cosmos.internal.directconnectivity.rntbd.RntbdResponse.lambda$release$0(RntbdResponse.java:198)
java.util.concurrent.atomic.AtomicInteger.accumulateAndGet(AtomicInteger.java:289)
com.azure.data.cosmos.internal.directconnectivity.rntbd.RntbdResponse.release(RntbdResponse.java:184)
com.azure.data.cosmos.internal.directconnectivity.rntbd.RntbdResponse.release(RntbdResponse.java:172)
com.azure.data.cosmos.internal.directconnectivity.rntbd.RntbdRequestManager.channelRead(RntbdRequestManager.java:196)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321)
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436)
io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:253)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1470)
io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1219)
io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1266)
io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:498)
io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:437)
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:748)
#2:
com.azure.data.cosmos.internal.directconnectivity.rntbd.RntbdToken.getValue(RntbdToken.java:85)
com.azure.data.cosmos.internal.directconnectivity.rntbd.RntbdToken.getValue(RntbdToken.java:95)
com.azure.data.cosmos.internal.directconnectivity.rntbd.RntbdResponseHeaders.toIntegerEntry(RntbdResponseHeaders.java:496)
com.azure.data.cosmos.internal.directconnectivity.rntbd.RntbdResponseHeaders.lambda$collectEntries$44(RntbdResponseHeaders.java:469)
com.azure.data.cosmos.internal.directconnectivity.rntbd.RntbdResponseHeaders.lambda$asList$0(RntbdResponseHeaders.java:203)
com.azure.data.cosmos.internal.directconnectivity.rntbd.RntbdResponseHeaders.collectEntries(RntbdResponseHeaders.java:468)
com.azure.data.cosmos.internal.directconnectivity.rntbd.RntbdResponseHeaders.asList(RntbdResponseHeaders.java:201)
com.azure.data.cosmos.internal.directconnectivity.rntbd.RntbdResponse.toStoreResponse(RntbdResponse.java:239)
com.azure.data.cosmos.internal.directconnectivity.rntbd.RntbdRequestManager.messageReceived(RntbdRequestManager.java:707)
com.azure.data.cosmos.internal.directconnectivity.rntbd.RntbdRequestManager.channelRead(RntbdRequestManager.java:175)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321)
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436)
io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:253)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1470)
io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1219)
io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1266)
io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:498)
io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:437)
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:748)
#3:
io.netty.buffer.AdvancedLeakAwareByteBuf.readByte(AdvancedLeakAwareByteBuf.java:400)
com.azure.data.cosmos.internal.directconnectivity.rntbd.RntbdTokenStream.decode(RntbdTokenStream.java:68)
com.azure.data.cosmos.internal.directconnectivity.rntbd.RntbdResponseHeaders.decode(RntbdResponseHeaders.java:227)
com.azure.data.cosmos.internal.directconnectivity.rntbd.RntbdResponse.decode(RntbdResponse.java:104)
com.azure.data.cosmos.internal.directconnectivity.rntbd.RntbdResponseDecoder.decode(RntbdResponseDecoder.java:32)
io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:498)
io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:437)
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436)
io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:253)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1470)
io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1219)
io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1266)
io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:498)
io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:437)
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:748)
#4:
io.netty.buffer.AdvancedLeakAwareByteBuf.readShortLE(AdvancedLeakAwareByteBuf.java:826)
com.azure.data.cosmos.internal.directconnectivity.rntbd.RntbdTokenStream.decode(RntbdTokenStream.java:67)
com.azure.data.cosmos.internal.directconnectivity.rntbd.RntbdResponseHeaders.decode(RntbdResponseHeaders.java:227)
com.azure.data.cosmos.internal.directconnectivity.rntbd.RntbdResponse.decode(RntbdResponse.java:104)
com.azure.data.cosmos.internal.directconnectivity.rntbd.RntbdResponseDecoder.decode(RntbdResponseDecoder.java:32)
io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:498)
io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:437)
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436)
io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:253)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1470)
io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1219)
io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1266)
io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:498)
io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:437)
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:748)
#5:
io.netty.buffer.AdvancedLeakAwareByteBuf.getUnsignedByte(AdvancedLeakAwareByteBuf.java:160)
com.azure.data.cosmos.internal.directconnectivity.rntbd.RntbdTokenType$RntbdShortString.readSlice(RntbdTokenType.java:671)
com.azure.data.cosmos.internal.directconnectivity.rntbd.RntbdToken.decode(RntbdToken.java:156)
com.azure.data.cosmos.internal.directconnectivity.rntbd.RntbdTokenStream.decode(RntbdTokenStream.java:76)
com.azure.data.cosmos.internal.directconnectivity.rntbd.RntbdResponseHeaders.decode(RntbdResponseHeaders.java:227)
com.azure.data.cosmos.internal.directconnectivity.rntbd.RntbdResponse.decode(RntbdResponse.java:104)
com.azure.data.cosmos.internal.directconnectivity.rntbd.RntbdResponseDecoder.decode(RntbdResponseDecoder.java:32)
io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:498)
io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:437)
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436)
io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:253)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1470)
io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1219)
io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1266)
io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:498)
io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:437)
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:748)
#6:
io.netty.buffer.AdvancedLeakAwareByteBuf.readSlice(AdvancedLeakAwareByteBuf.java:118)
com.azure.data.cosmos.internal.directconnectivity.rntbd.RntbdTokenType$RntbdByte.readSlice(RntbdTokenType.java:150)
com.azure.data.cosmos.internal.directconnectivity.rntbd.RntbdToken.decode(RntbdToken.java:156)
com.azure.data.cosmos.internal.directconnectivity.rntbd.RntbdTokenStream.decode(RntbdTokenStream.java:76)
com.azure.data.cosmos.internal.directconnectivity.rntbd.RntbdResponseHeaders.decode(RntbdResponseHeaders.java:227)
com.azure.data.cosmos.internal.directconnectivity.rntbd.RntbdResponse.decode(RntbdResponse.java:104)
com.azure.data.cosmos.internal.directconnectivity.rntbd.RntbdResponseDecoder.decode(RntbdResponseDecoder.java:32)
io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:498)
io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:437)
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436)
io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:253)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1470)
io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1219)
io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1266)
io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:498)
io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:437)
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:748)
#7:
io.netty.buffer.AdvancedLeakAwareByteBuf.readShortLE(AdvancedLeakAwareByteBuf.java:826)
com.azure.data.cosmos.internal.directconnectivity.rntbd.RntbdUUID.decode(RntbdUUID.java:52)
com.azure.data.cosmos.internal.directconnectivity.rntbd.RntbdResponseStatus.decode(RntbdResponseStatus.java:80)
com.azure.data.cosmos.internal.directconnectivity.rntbd.RntbdResponse.decode(RntbdResponse.java:103)
com.azure.data.cosmos.internal.directconnectivity.rntbd.RntbdResponseDecoder.decode(RntbdResponseDecoder.java:32)
io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:498)
io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:437)
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436)
io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:253)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1470)
io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1219)
io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1266)
io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:498)
io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:437)
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:748)
#8:
Hint: 'RntbdResponseDecoder#0' will handle the message from this point.
io.netty.channel.DefaultChannelPipeline.touch(DefaultChannelPipeline.java:116)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436)
io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:253)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1470)
io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1219)
io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1266)
io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:498)
io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:437)
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:748)
#9:
Hint: 'IdleStateHandler#0' will handle the message from this point.
io.netty.channel.DefaultChannelPipeline.touch(DefaultChannelPipeline.java:116)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1470)
io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1219)
io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1266)
io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:498)
io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:437)
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:748)
#10:
io.netty.buffer.AdvancedLeakAwareByteBuf.internalNioBuffer(AdvancedLeakAwareByteBuf.java:736)
io.netty.handler.ssl.SslHandler.toByteBuffer(SslHandler.java:1480)
io.netty.handler.ssl.SslHandler.access$300(SslHandler.java:166)
io.netty.handler.ssl.SslHandler$SslEngineType$3.unwrap(SslHandler.java:282)
io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1324)
io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1219)
io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1266)
io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:498)
io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:437)
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:748)
#11:
io.netty.buffer.AdvancedLeakAwareByteBuf.nioBufferCount(AdvancedLeakAwareByteBuf.java:706)
io.netty.handler.ssl.SslHandler.toByteBuffer(SslHandler.java:1480)
io.netty.handler.ssl.SslHandler.access$300(SslHandler.java:166)
io.netty.handler.ssl.SslHandler$SslEngineType$3.unwrap(SslHandler.java:282)
io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1324)
io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1219)
io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1266)
io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:498)
io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:437)
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:748)
Created at:
io.netty.buffer.PooledByteBufAllocator.newHeapBuffer(PooledByteBufAllocator.java:332)
io.netty.buffer.AbstractByteBufAllocator.heapBuffer(AbstractByteBufAllocator.java:168)
io.netty.buffer.AbstractByteBufAllocator.heapBuffer(AbstractByteBufAllocator.java:159)
io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:117)
io.netty.handler.ssl.SslHandler.allocate(SslHandler.java:2136)
io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1319)
io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1219)
io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1266)
io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:498)
io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:437)
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:748)
: 144 leak records were discarded because the leak record count is targeted to 4. Use system property io.netty.leakDetection.targetRecords to increase the limit.
To Reproduce Steps to reproduce the behavior: Unfortunately, I have not been able to reproduce this in isolation, only in production, as it seems to require a certain amount of sustained traffic to happen.
Screenshots
Heap usage during leaking behavior:
Heap usage after switching to GATEWAY mode:
Setup (please complete the following information):
- OS: openjdk:8-jdk-alpine
- Version of the Library used: 3.5.0/3.7.0
We are running our services in AKS using open JDK 8
Gradle Runtime Dependency Report
+--- com.redacted.redacted:redacted-common:1.0.390
| +--- org.jetbrains.kotlinx:kotlinx-coroutines-core:1.3.3
| | +--- org.jetbrains.kotlin:kotlin-stdlib:1.3.61 -> 1.3.70
| | | +--- org.jetbrains.kotlin:kotlin-stdlib-common:1.3.70
| | | \--- org.jetbrains:annotations:13.0
| | \--- org.jetbrains.kotlin:kotlin-stdlib-common:1.3.61 -> 1.3.70
| +--- org.jetbrains.kotlinx:kotlinx-coroutines-reactor:1.3.3
| | +--- org.jetbrains.kotlin:kotlin-stdlib:1.3.61 -> 1.3.70 (*)
| | +--- org.jetbrains.kotlinx:kotlinx-coroutines-core:1.3.3 (*)
| | +--- io.projectreactor:reactor-core:3.2.5.RELEASE -> 3.3.3.RELEASE
| | | \--- org.reactivestreams:reactive-streams:1.0.3
| | \--- org.jetbrains.kotlinx:kotlinx-coroutines-reactive:1.3.3
| | +--- org.jetbrains.kotlin:kotlin-stdlib:1.3.61 -> 1.3.70 (*)
| | +--- org.jetbrains.kotlinx:kotlinx-coroutines-core:1.3.3 (*)
| | \--- org.reactivestreams:reactive-streams:1.0.2 -> 1.0.3
| +--- org.jetbrains.kotlinx:kotlinx-coroutines-slf4j:1.3.3
| | +--- org.jetbrains.kotlin:kotlin-stdlib:1.3.61 -> 1.3.70 (*)
| | +--- org.jetbrains.kotlinx:kotlinx-coroutines-core:1.3.3 (*)
| | \--- org.slf4j:slf4j-api:1.7.25 -> 1.7.30
| +--- org.jetbrains.kotlinx:kotlinx-coroutines-rx2:1.3.3
| | +--- org.jetbrains.kotlin:kotlin-stdlib:1.3.61 -> 1.3.70 (*)
| | +--- org.jetbrains.kotlinx:kotlinx-coroutines-core:1.3.3 (*)
| | +--- org.jetbrains.kotlinx:kotlinx-coroutines-reactive:1.3.3 (*)
| | \--- io.reactivex.rxjava2:rxjava:2.2.8 -> 2.2.17
| | \--- org.reactivestreams:reactive-streams:1.0.3
| +--- org.jetbrains.kotlinx:kotlinx-coroutines-jdk8:1.3.3
| | +--- org.jetbrains.kotlin:kotlin-stdlib:1.3.61 -> 1.3.70 (*)
| | \--- org.jetbrains.kotlinx:kotlinx-coroutines-core:1.3.3 (*)
| +--- com.fasterxml.jackson.module:jackson-module-kotlin:2.10.3 -> 2.10.2
| | +--- com.fasterxml.jackson.core:jackson-databind:2.10.2
| | | +--- com.fasterxml.jackson.core:jackson-annotations:2.10.2
| | | \--- com.fasterxml.jackson.core:jackson-core:2.10.2
| | +--- com.fasterxml.jackson.core:jackson-annotations:2.10.2
| | \--- org.jetbrains.kotlin:kotlin-reflect:1.3.61 -> 1.3.70
| | \--- org.jetbrains.kotlin:kotlin-stdlib:1.3.70 (*)
| +--- com.microsoft.azure:azure-servicebus-spring-boot-starter:2.2.1
| | +--- org.springframework.boot:spring-boot-starter:2.2.0.RELEASE -> 2.2.4.RELEASE
| | | +--- org.springframework.boot:spring-boot:2.2.4.RELEASE
| | | | +--- org.springframework:spring-core:5.2.3.RELEASE
| | | | | \--- org.springframework:spring-jcl:5.2.3.RELEASE
| | | | \--- org.springframework:spring-context:5.2.3.RELEASE
| | | | +--- org.springframework:spring-aop:5.2.3.RELEASE
| | | | | +--- org.springframework:spring-beans:5.2.3.RELEASE
| | | | | | \--- org.springframework:spring-core:5.2.3.RELEASE (*)
| | | | | \--- org.springframework:spring-core:5.2.3.RELEASE (*)
| | | | +--- org.springframework:spring-beans:5.2.3.RELEASE (*)
| | | | +--- org.springframework:spring-core:5.2.3.RELEASE (*)
| | | | \--- org.springframework:spring-expression:5.2.3.RELEASE
| | | | \--- org.springframework:spring-core:5.2.3.RELEASE (*)
| | | +--- org.springframework.boot:spring-boot-autoconfigure:2.2.4.RELEASE
| | | | \--- org.springframework.boot:spring-boot:2.2.4.RELEASE (*)
| | | +--- org.springframework.boot:spring-boot-starter-logging:2.2.4.RELEASE
| | | | +--- ch.qos.logback:logback-classic:1.2.3
| | | | | +--- ch.qos.logback:logback-core:1.2.3
| | | | | \--- org.slf4j:slf4j-api:1.7.25 -> 1.7.30
| | | | +--- org.apache.logging.log4j:log4j-to-slf4j:2.12.1
| | | | | +--- org.slf4j:slf4j-api:1.7.25 -> 1.7.30
| | | | | \--- org.apache.logging.log4j:log4j-api:2.12.1
| | | | \--- org.slf4j:jul-to-slf4j:1.7.30
| | | | \--- org.slf4j:slf4j-api:1.7.30
| | | +--- jakarta.annotation:jakarta.annotation-api:1.3.5
| | | +--- org.springframework:spring-core:5.2.3.RELEASE (*)
| | | \--- org.yaml:snakeyaml:1.25
| | +--- org.springframework.boot:spring-boot-starter-validation:2.2.0.RELEASE -> 2.2.4.RELEASE
| | | +--- org.springframework.boot:spring-boot-starter:2.2.4.RELEASE (*)
| | | +--- jakarta.validation:jakarta.validation-api:2.0.2
| | | +--- org.apache.tomcat.embed:tomcat-embed-el:9.0.30
| | | \--- org.hibernate.validator:hibernate-validator:6.0.18.Final
| | | +--- org.jboss.logging:jboss-logging:3.3.2.Final -> 3.4.1.Final
| | | \--- com.fasterxml:classmate:1.3.4 -> 1.5.1
| | +--- com.microsoft.azure:azure-spring-boot:2.2.1
| | | +--- org.springframework.boot:spring-boot-autoconfigure:2.2.0.RELEASE -> 2.2.4.RELEASE (*)
| | | +--- org.slf4j:slf4j-api:1.7.28 -> 1.7.30
| | | +--- org.springframework:spring-web:5.2.0.RELEASE -> 5.2.3.RELEASE
| | | | +--- org.springframework:spring-beans:5.2.3.RELEASE (*)
| | | | \--- org.springframework:spring-core:5.2.3.RELEASE (*)
| | | \--- com.fasterxml.jackson.core:jackson-databind:2.10.0 -> 2.10.2 (*)
| | +--- com.microsoft.azure:azure-servicebus:1.2.15 -> 1.2.18
| | | +--- org.apache.qpid:proton-j:0.31.0
| | | +--- org.slf4j:slf4j-api:1.7.0 -> 1.7.30
| | | \--- com.microsoft.azure:adal4j:1.3.0
| | | +--- com.nimbusds:oauth2-oidc-sdk:5.24.1
| | | | +--- javax.mail:mail:1.4.7
| | | | | \--- javax.activation:activation:1.1
| | | | +--- com.github.stephenc.jcip:jcip-annotations:1.0-1
| | | | +--- org.apache.commons:commons-lang3:3.5 -> 3.9
| | | | +--- org.apache.commons:commons-collections4:4.1 -> 4.2
| | | | +--- net.minidev:json-smart:1.3.1 -> 2.3
| | | | | \--- net.minidev:accessors-smart:1.2
| | | | | \--- org.ow2.asm:asm:5.0.4
| | | | +--- com.nimbusds:lang-tag:[1.4.3,) -> 1.4.4
| | | | \--- com.nimbusds:nimbus-jose-jwt:[4.29,) -> 8.9
| | | | +--- com.github.stephenc.jcip:jcip-annotations:1.0-1
| | | | \--- net.minidev:json-smart:[1.3.1,2.3] -> 2.3 (*)
| | | +--- com.google.code.gson:gson:2.2.4 -> 2.8.6
| | | +--- org.slf4j:slf4j-api:1.7.5 -> 1.7.30
| | | \--- commons-codec:commons-codec:1.10 -> 1.13
| | +--- com.google.code.findbugs:annotations:2.0.1
| | +--- com.fasterxml.jackson.core:jackson-core:2.10.0 -> 2.10.2
| | \--- commons-codec:commons-codec:1.13
| +--- com.microsoft.azure:azure-servicebus:1.2.18 (*)
| +--- com.microsoft.azure:applicationinsights-logging-logback:2.5.1
| | +--- com.microsoft.azure:applicationinsights-core:2.5.1
| | +--- ch.qos.logback:logback-classic:1.2.3 (*)
| | \--- ch.qos.logback:logback-core:1.2.3
| +--- org.springframework.boot:spring-boot-starter-security:2.2.4.RELEASE
| | +--- org.springframework.boot:spring-boot-starter:2.2.4.RELEASE (*)
| | +--- org.springframework:spring-aop:5.2.3.RELEASE (*)
| | +--- org.springframework.security:spring-security-config:5.2.1.RELEASE
| | | +--- org.springframework.security:spring-security-core:5.2.1.RELEASE
| | | | +--- org.springframework:spring-aop:5.2.1.RELEASE -> 5.2.3.RELEASE (*)
| | | | +--- org.springframework:spring-beans:5.2.1.RELEASE -> 5.2.3.RELEASE (*)
| | | | +--- org.springframework:spring-context:5.2.1.RELEASE -> 5.2.3.RELEASE (*)
| | | | +--- org.springframework:spring-core:5.2.1.RELEASE -> 5.2.3.RELEASE (*)
| | | | \--- org.springframework:spring-expression:5.2.1.RELEASE -> 5.2.3.RELEASE (*)
| | | +--- org.springframework:spring-aop:5.2.1.RELEASE -> 5.2.3.RELEASE (*)
| | | +--- org.springframework:spring-beans:5.2.1.RELEASE -> 5.2.3.RELEASE (*)
| | | +--- org.springframework:spring-context:5.2.1.RELEASE -> 5.2.3.RELEASE (*)
| | | \--- org.springframework:spring-core:5.2.1.RELEASE -> 5.2.3.RELEASE (*)
| | \--- org.springframework.security:spring-security-web:5.2.1.RELEASE
| | +--- org.springframework.security:spring-security-core:5.2.1.RELEASE (*)
| | +--- org.springframework:spring-aop:5.2.1.RELEASE -> 5.2.3.RELEASE (*)
| | +--- org.springframework:spring-beans:5.2.1.RELEASE -> 5.2.3.RELEASE (*)
| | +--- org.springframework:spring-context:5.2.1.RELEASE -> 5.2.3.RELEASE (*)
| | +--- org.springframework:spring-core:5.2.1.RELEASE -> 5.2.3.RELEASE (*)
| | +--- org.springframework:spring-expression:5.2.1.RELEASE -> 5.2.3.RELEASE (*)
| | \--- org.springframework:spring-web:5.2.1.RELEASE -> 5.2.3.RELEASE (*)
| +--- io.projectreactor:reactor-core:3.3.3.RELEASE (*)
| +--- io.projectreactor.addons:reactor-adapter:3.3.2.RELEASE
| | \--- io.projectreactor:reactor-core:3.3.2.RELEASE -> 3.3.3.RELEASE (*)
| +--- io.projectreactor.addons:reactor-extra:3.3.2.RELEASE
| | \--- io.projectreactor:reactor-core:3.3.2.RELEASE -> 3.3.3.RELEASE (*)
| +--- io.projectreactor.kotlin:reactor-kotlin-extensions:1.0.2.RELEASE
| | \--- io.projectreactor:reactor-core:3.3.2.RELEASE -> 3.3.3.RELEASE (*)
| +--- org.jetbrains.kotlin:kotlin-stdlib:1.3.70 (*)
| +--- org.jetbrains.kotlin:kotlin-reflect:1.3.70 (*)
| +--- net.minidev:json-smart:2.3 (*)
| +--- com.microsoft.azure:azure-eventgrid:1.3.0
| | \--- com.microsoft.azure:azure-client-runtime:1.5.0 -> 1.6.12
| | \--- com.microsoft.rest:client-runtime:1.6.12
| | +--- com.google.guava:guava:20.0 -> 28.2-android
| | | +--- com.google.guava:failureaccess:1.0.1
| | | +--- com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
| | | +--- com.google.code.findbugs:jsr305:3.0.2
| | | +--- org.checkerframework:checker-compat-qual:2.5.5
| | | +--- com.google.errorprone:error_prone_annotations:2.3.4
| | | \--- com.google.j2objc:j2objc-annotations:1.3
| | +--- com.squareup.retrofit2:retrofit:2.5.0
| | | \--- com.squareup.okhttp3:okhttp:3.12.0 -> 3.12.2
| | | \--- com.squareup.okio:okio:1.15.0
| | +--- com.squareup.okhttp3:okhttp:3.12.2 (*)
| | +--- com.squareup.okhttp3:logging-interceptor:3.12.2 -> 3.12.0
| | | \--- com.squareup.okhttp3:okhttp:3.12.0 -> 3.12.2 (*)
| | +--- com.squareup.okhttp3:okhttp-urlconnection:3.12.2 -> 3.14.6
| | | \--- com.squareup.okhttp3:okhttp:3.14.6 -> 3.12.2 (*)
| | +--- com.squareup.retrofit2:converter-jackson:2.5.0
| | | +--- com.squareup.retrofit2:retrofit:2.5.0 (*)
| | | \--- com.fasterxml.jackson.core:jackson-databind:2.9.4 -> 2.10.2 (*)
| | +--- com.fasterxml.jackson.datatype:jackson-datatype-joda:2.9.8 -> 2.10.2
| | | +--- com.fasterxml.jackson.core:jackson-annotations:2.10.2
| | | +--- com.fasterxml.jackson.core:jackson-core:2.10.2
| | | +--- com.fasterxml.jackson.core:jackson-databind:2.10.2 (*)
| | | \--- joda-time:joda-time:2.9.9 -> 2.10.5
| | +--- org.apache.commons:commons-lang3:3.4 -> 3.9
| | +--- io.reactivex:rxjava:1.3.8
| | +--- com.squareup.retrofit2:adapter-rxjava:2.4.0
| | | +--- com.squareup.retrofit2:retrofit:2.4.0 -> 2.5.0 (*)
| | | \--- io.reactivex:rxjava:1.3.0 -> 1.3.8
| | +--- org.slf4j:slf4j-api:1.7.22 -> 1.7.30
| | \--- com.microsoft.azure:azure-annotations:1.7.0
| +--- com.microsoft.azure:azure-cosmos:3.7.0
| | +--- com.fasterxml.jackson.core:jackson-databind:2.10.1 -> 2.10.2 (*)
| | +--- com.fasterxml.jackson.datatype:jackson-datatype-jsr310:2.10.1 -> 2.10.2
| | | +--- com.fasterxml.jackson.core:jackson-annotations:2.10.2
| | | +--- com.fasterxml.jackson.core:jackson-core:2.10.2
| | | \--- com.fasterxml.jackson.core:jackson-databind:2.10.2 (*)
| | +--- com.fasterxml.uuid:java-uuid-generator:3.2.0
| | +--- commons-io:commons-io:2.5
| | +--- commons-validator:commons-validator:1.6
| | | +--- commons-digester:commons-digester:1.8.1
| | | +--- commons-logging:commons-logging:1.2
| | | \--- commons-collections:commons-collections:3.2.2
| | +--- io.projectreactor:reactor-core:3.3.0.RELEASE -> 3.3.3.RELEASE (*)
| | +--- io.netty:netty-codec-http:4.1.44.Final -> 4.1.45.Final
| | | +--- io.netty:netty-common:4.1.45.Final
| | | +--- io.netty:netty-buffer:4.1.45.Final
| | | | \--- io.netty:netty-common:4.1.45.Final
| | | +--- io.netty:netty-transport:4.1.45.Final
| | | | +--- io.netty:netty-common:4.1.45.Final
| | | | +--- io.netty:netty-buffer:4.1.45.Final (*)
| | | | \--- io.netty:netty-resolver:4.1.45.Final
| | | | \--- io.netty:netty-common:4.1.45.Final
| | | +--- io.netty:netty-codec:4.1.45.Final
| | | | +--- io.netty:netty-common:4.1.45.Final
| | | | +--- io.netty:netty-buffer:4.1.45.Final (*)
| | | | \--- io.netty:netty-transport:4.1.45.Final (*)
| | | \--- io.netty:netty-handler:4.1.45.Final
| | | +--- io.netty:netty-common:4.1.45.Final
| | | +--- io.netty:netty-buffer:4.1.45.Final (*)
| | | +--- io.netty:netty-transport:4.1.45.Final (*)
| | | \--- io.netty:netty-codec:4.1.45.Final (*)
| | +--- io.netty:netty-handler:4.1.44.Final -> 4.1.45.Final (*)
| | +--- org.slf4j:slf4j-api:1.7.28 -> 1.7.30
| | +--- org.apache.commons:commons-lang3:3.8.1 -> 3.9
| | +--- org.apache.commons:commons-collections4:4.2
| | +--- org.apache.commons:commons-text:1.6
| | | \--- org.apache.commons:commons-lang3:3.8.1 -> 3.9
| | +--- com.google.guava:guava:25.0-jre -> 28.2-android (*)
| | +--- io.dropwizard.metrics:metrics-core:4.1.0 -> 4.1.2
| | | \--- org.slf4j:slf4j-api:1.7.29 -> 1.7.30
| | +--- io.micrometer:micrometer-core:1.2.0 -> 1.3.2
| | | +--- org.hdrhistogram:HdrHistogram:2.1.11
| | | \--- org.latencyutils:LatencyUtils:2.0.3
| | +--- io.reactivex.rxjava2:rxjava:2.2.4 -> 2.2.17 (*)
| | \--- io.projectreactor.netty:reactor-netty:0.9.4.RELEASE
| | +--- io.netty:netty-codec-http:4.1.44.Final -> 4.1.45.Final (*)
| | +--- io.netty:netty-codec-http2:4.1.44.Final -> 4.1.45.Final
| | | +--- io.netty:netty-common:4.1.45.Final
| | | +--- io.netty:netty-buffer:4.1.45.Final (*)
| | | +--- io.netty:netty-transport:4.1.45.Final (*)
| | | +--- io.netty:netty-codec:4.1.45.Final (*)
| | | +--- io.netty:netty-handler:4.1.45.Final (*)
| | | \--- io.netty:netty-codec-http:4.1.45.Final (*)
| | +--- io.netty:netty-handler:4.1.44.Final -> 4.1.45.Final (*)
| | +--- io.netty:netty-handler-proxy:4.1.44.Final -> 4.1.45.Final
| | | +--- io.netty:netty-common:4.1.45.Final
| | | +--- io.netty:netty-buffer:4.1.45.Final (*)
| | | +--- io.netty:netty-transport:4.1.45.Final (*)
| | | +--- io.netty:netty-codec:4.1.45.Final (*)
| | | +--- io.netty:netty-codec-socks:4.1.45.Final
| | | | +--- io.netty:netty-common:4.1.45.Final
| | | | +--- io.netty:netty-buffer:4.1.45.Final (*)
| | | | +--- io.netty:netty-transport:4.1.45.Final (*)
| | | | \--- io.netty:netty-codec:4.1.45.Final (*)
| | | \--- io.netty:netty-codec-http:4.1.45.Final (*)
| | +--- io.netty:netty-transport-native-epoll:4.1.44.Final -> 4.1.45.Final
| | | +--- io.netty:netty-common:4.1.45.Final
| | | +--- io.netty:netty-buffer:4.1.45.Final (*)
| | | +--- io.netty:netty-transport:4.1.45.Final (*)
| | | \--- io.netty:netty-transport-native-unix-common:4.1.45.Final
| | | +--- io.netty:netty-common:4.1.45.Final
| | | +--- io.netty:netty-buffer:4.1.45.Final (*)
| | | \--- io.netty:netty-transport:4.1.45.Final (*)
| | \--- io.projectreactor:reactor-core:3.3.2.RELEASE -> 3.3.3.RELEASE (*)
| +--- com.microsoft.azure:applicationinsights-spring-boot-starter:2.5.1
| | +--- com.microsoft.azure:applicationinsights-core:2.5.1
| | \--- com.microsoft.azure:applicationinsights-web:2.5.1
| +--- com.microsoft.azure:azure-keyvault:1.2.2
| | +--- com.microsoft.azure:azure-keyvault-cryptography:1.2.2
| | | +--- com.microsoft.azure:azure-keyvault-webkey:1.2.2
| | | | +--- com.fasterxml.jackson.core:jackson-databind:2.9.9 -> 2.10.2 (*)
| | | | +--- com.fasterxml.jackson.core:jackson-core:2.9.9 -> 2.10.2
| | | | +--- com.fasterxml.jackson.core:jackson-annotations:2.9.9 -> 2.10.2
| | | | +--- commons-codec:commons-codec:1.10 -> 1.13
| | | | \--- com.google.guava:guava:24.1.1-jre -> 28.2-android (*)
| | | +--- com.microsoft.azure:azure-keyvault-core:1.2.2
| | | | +--- org.apache.commons:commons-lang3:3.8.1 -> 3.9
| | | | \--- com.google.guava:guava:24.1.1-jre -> 28.2-android (*)
| | | +--- com.fasterxml.jackson.core:jackson-databind:2.9.9 -> 2.10.2 (*)
| | | +--- org.apache.commons:commons-lang3:3.8.1 -> 3.9
| | | +--- commons-codec:commons-codec:1.10 -> 1.13
| | | \--- com.google.guava:guava:24.1.1-jre -> 28.2-android (*)
| | +--- com.microsoft.azure:azure-keyvault-webkey:1.2.2 (*)
| | +--- com.microsoft.azure:azure-client-runtime:1.6.12 (*)
| | \--- com.microsoft.rest:client-runtime:1.6.12 (*)
| +--- org.springframework.boot:spring-boot-starter-actuator:2.2.4.RELEASE
| | +--- org.springframework.boot:spring-boot-starter:2.2.4.RELEASE (*)
| | +--- org.springframework.boot:spring-boot-actuator-autoconfigure:2.2.4.RELEASE
| | | +--- org.springframework.boot:spring-boot-actuator:2.2.4.RELEASE
| | | | +--- org.springframework.boot:spring-boot:2.2.4.RELEASE (*)
| | | | \--- com.fasterxml.jackson.datatype:jackson-datatype-jsr310:2.10.2 (*)
| | | +--- org.springframework.boot:spring-boot-autoconfigure:2.2.4.RELEASE (*)
| | | +--- com.fasterxml.jackson.core:jackson-databind:2.10.2 (*)
| | | +--- org.springframework:spring-core:5.2.3.RELEASE (*)
| | | +--- org.springframework:spring-context:5.2.3.RELEASE (*)
| | | \--- com.fasterxml.jackson.datatype:jackson-datatype-jsr310:2.10.2 (*)
| | \--- io.micrometer:micrometer-core:1.3.2 (*)
| +--- org.springframework.boot:spring-boot-starter-webflux:2.2.4.RELEASE
| | +--- org.springframework.boot:spring-boot-starter:2.2.4.RELEASE (*)
| | +--- org.springframework.boot:spring-boot-starter-json:2.2.4.RELEASE
| | | +--- org.springframework.boot:spring-boot-starter:2.2.4.RELEASE (*)
| | | +--- org.springframework:spring-web:5.2.3.RELEASE (*)
| | | +--- com.fasterxml.jackson.core:jackson-databind:2.10.2 (*)
| | | +--- com.fasterxml.jackson.datatype:jackson-datatype-jdk8:2.10.2
| | | | +--- com.fasterxml.jackson.core:jackson-core:2.10.2
| | | | \--- com.fasterxml.jackson.core:jackson-databind:2.10.2 (*)
| | | +--- com.fasterxml.jackson.datatype:jackson-datatype-jsr310:2.10.2 (*)
| | | \--- com.fasterxml.jackson.module:jackson-module-parameter-names:2.10.2
| | | +--- com.fasterxml.jackson.core:jackson-core:2.10.2
| | | \--- com.fasterxml.jackson.core:jackson-databind:2.10.2 (*)
| | +--- org.springframework.boot:spring-boot-starter-reactor-netty:2.2.4.RELEASE
| | | +--- io.projectreactor.netty:reactor-netty:0.9.4.RELEASE (*)
| | | \--- org.glassfish:jakarta.el:3.0.3
| | +--- org.springframework.boot:spring-boot-starter-validation:2.2.4.RELEASE (*)
| | +--- org.springframework:spring-web:5.2.3.RELEASE (*)
| | +--- org.springframework:spring-webflux:5.2.3.RELEASE
| | | +--- org.springframework:spring-beans:5.2.3.RELEASE (*)
| | | +--- org.springframework:spring-core:5.2.3.RELEASE (*)
| | | +--- org.springframework:spring-web:5.2.3.RELEASE (*)
| | | \--- io.projectreactor:reactor-core:3.3.2.RELEASE -> 3.3.3.RELEASE (*)
| | \--- org.synchronoss.cloud:nio-multipart-parser:1.1.0
| | +--- org.slf4j:slf4j-api:1.7.12 -> 1.7.30
| | \--- org.synchronoss.cloud:nio-stream-storage:1.1.3
| | \--- org.slf4j:slf4j-api:1.7.12 -> 1.7.30
| +--- io.springfox:springfox-swagger2:3.0.0-SNAPSHOT
| | +--- io.swagger:swagger-annotations:1.5.20
| | +--- io.swagger:swagger-models:1.5.20
| | | +--- com.fasterxml.jackson.core:jackson-annotations:2.9.5 -> 2.10.2
| | | +--- org.slf4j:slf4j-api:1.7.22 -> 1.7.30
| | | \--- io.swagger:swagger-annotations:1.5.20
| | +--- io.springfox:springfox-spi:3.0.0-SNAPSHOT
| | | \--- io.springfox:springfox-core:3.0.0-SNAPSHOT
| | | +--- net.bytebuddy:byte-buddy:1.9.10 -> 1.10.6
| | | +--- com.fasterxml:classmate:1.4.0 -> 1.5.1
| | | +--- org.slf4j:slf4j-api:1.7.25 -> 1.7.30
| | | +--- org.springframework.plugin:spring-plugin-core:2.0.0.RELEASE
| | | | +--- org.springframework:spring-beans:5.2.0.RELEASE -> 5.2.3.RELEASE (*)
| | | | +--- org.springframework:spring-context:5.2.0.RELEASE -> 5.2.3.RELEASE (*)
| | | | +--- org.springframework:spring-aop:5.2.0.RELEASE -> 5.2.3.RELEASE (*)
| | | | \--- org.slf4j:slf4j-api:1.7.25 -> 1.7.30
| | | \--- org.springframework.plugin:spring-plugin-metadata:2.0.0.RELEASE
| | | +--- org.springframework.plugin:spring-plugin-core:2.0.0.RELEASE (*)
| | | \--- org.slf4j:slf4j-api:1.7.25 -> 1.7.30
| | +--- io.springfox:springfox-schema:3.0.0-SNAPSHOT
| | | +--- io.springfox:springfox-core:3.0.0-SNAPSHOT (*)
| | | \--- io.springfox:springfox-spi:3.0.0-SNAPSHOT (*)
| | +--- io.springfox:springfox-swagger-common:3.0.0-SNAPSHOT
| | | +--- io.swagger:swagger-annotations:1.5.20
| | | +--- io.swagger:swagger-models:1.5.20 (*)
| | | +--- io.springfox:springfox-spi:3.0.0-SNAPSHOT (*)
| | | +--- io.springfox:springfox-schema:3.0.0-SNAPSHOT (*)
| | | +--- io.springfox:springfox-spring-web:3.0.0-SNAPSHOT
| | | | +--- io.github.classgraph:classgraph:4.1.7
| | | | +--- com.fasterxml:classmate:1.4.0 -> 1.5.1
| | | | +--- org.slf4j:slf4j-api:1.7.25 -> 1.7.30
| | | | +--- org.springframework.plugin:spring-plugin-core:2.0.0.RELEASE (*)
| | | | +--- org.springframework.plugin:spring-plugin-metadata:2.0.0.RELEASE (*)
| | | | \--- io.springfox:springfox-spi:3.0.0-SNAPSHOT (*)
| | | +--- com.fasterxml:classmate:1.4.0 -> 1.5.1
| | | +--- org.slf4j:slf4j-api:1.7.25 -> 1.7.30
| | | +--- org.springframework.plugin:spring-plugin-core:2.0.0.RELEASE (*)
| | | \--- org.springframework.plugin:spring-plugin-metadata:2.0.0.RELEASE (*)
| | +--- io.springfox:springfox-spring-web:3.0.0-SNAPSHOT (*)
| | +--- com.fasterxml:classmate:1.4.0 -> 1.5.1
| | +--- org.slf4j:slf4j-api:1.7.25 -> 1.7.30
| | +--- org.springframework.plugin:spring-plugin-core:2.0.0.RELEASE (*)
| | +--- org.springframework.plugin:spring-plugin-metadata:2.0.0.RELEASE (*)
| | \--- org.mapstruct:mapstruct:1.3.1.Final
| +--- io.springfox:springfox-spring-webflux:3.0.0-SNAPSHOT
| | +--- com.fasterxml:classmate:1.4.0 -> 1.5.1
| | +--- org.slf4j:slf4j-api:1.7.25 -> 1.7.30
| | +--- org.springframework.plugin:spring-plugin-core:2.0.0.RELEASE (*)
| | +--- org.springframework.plugin:spring-plugin-metadata:2.0.0.RELEASE (*)
| | +--- io.springfox:springfox-spi:3.0.0-SNAPSHOT (*)
| | \--- io.springfox:springfox-spring-web:3.0.0-SNAPSHOT (*)
| +--- org.springframework.security:spring-security-oauth2-jose:5.3.0.RELEASE -> 5.2.1.RELEASE
| | +--- com.nimbusds:nimbus-jose-jwt:7.8 -> 8.9 (*)
| | +--- org.springframework.security:spring-security-core:5.2.1.RELEASE (*)
| | +--- org.springframework.security:spring-security-oauth2-core:5.2.1.RELEASE
| | | +--- org.springframework.security:spring-security-core:5.2.1.RELEASE (*)
| | | +--- org.springframework:spring-core:5.2.1.RELEASE -> 5.2.3.RELEASE (*)
| | | \--- org.springframework:spring-web:5.2.1.RELEASE -> 5.2.3.RELEASE (*)
| | \--- org.springframework:spring-core:5.2.1.RELEASE -> 5.2.3.RELEASE (*)
| +--- com.nimbusds:nimbus-jose-jwt:8.9 (*)
| +--- com.onelogin:java-saml:2.5.0
| | +--- com.onelogin:java-saml-core:2.5.0
| | | +--- org.slf4j:slf4j-api:1.7.12 -> 1.7.30
| | | +--- joda-time:joda-time:2.10.3 -> 2.10.5
| | | +--- org.apache.commons:commons-lang3:3.4 -> 3.9
| | | +--- org.apache.santuario:xmlsec:2.1.4
| | | | +--- org.slf4j:slf4j-api:1.7.26 -> 1.7.30
| | | | +--- com.fasterxml.woodstox:woodstox-core:5.0.3
| | | | | \--- org.codehaus.woodstox:stax2-api:3.1.4
| | | | \--- commons-codec:commons-codec:1.12 -> 1.13
| | | \--- commons-codec:commons-codec:1.12 -> 1.13
| | +--- org.slf4j:slf4j-api:1.7.12 -> 1.7.30
| | +--- joda-time:joda-time:2.10.3 -> 2.10.5
| | +--- org.apache.commons:commons-lang3:3.4 -> 3.9
| | +--- org.apache.santuario:xmlsec:2.1.4 (*)
| | \--- commons-codec:commons-codec:1.12 -> 1.13
| \--- io.reactivex:rxjava-reactive-streams:1.2.1
| +--- io.reactivex:rxjava:1.2.2 -> 1.3.8
| \--- org.reactivestreams:reactive-streams:1.0.0 -> 1.0.3
+--- org.jetbrains.kotlin:kotlin-stdlib:1.3.70 (*)
+--- org.jetbrains.kotlin:kotlin-reflect:1.3.70 (*)
+--- org.springframework.boot:spring-boot-starter-actuator:2.2.4.RELEASE (*)
+--- org.springframework.boot:spring-boot-starter-webflux:2.2.4.RELEASE (*)
+--- org.springframework.cloud:spring-cloud-starter-config:2.2.2.RELEASE
| +--- org.springframework.cloud:spring-cloud-starter:2.2.2.RELEASE
| | +--- org.springframework.boot:spring-boot-starter:2.2.5.RELEASE -> 2.2.4.RELEASE (*)
| | +--- org.springframework.cloud:spring-cloud-context:2.2.2.RELEASE
| | | \--- org.springframework.security:spring-security-crypto:5.2.2.RELEASE -> 5.2.1.RELEASE
| | +--- org.springframework.cloud:spring-cloud-commons:2.2.2.RELEASE
| | | \--- org.springframework.security:spring-security-crypto:5.2.2.RELEASE -> 5.2.1.RELEASE
| | \--- org.springframework.security:spring-security-rsa:1.0.9.RELEASE
| | \--- org.bouncycastle:bcpkix-jdk15on:1.64
| | \--- org.bouncycastle:bcprov-jdk15on:1.64
| +--- org.springframework.cloud:spring-cloud-config-client:2.2.2.RELEASE
| | +--- org.springframework.boot:spring-boot-autoconfigure:2.2.5.RELEASE -> 2.2.4.RELEASE (*)
| | +--- org.springframework.cloud:spring-cloud-commons:2.2.2.RELEASE (*)
| | +--- org.springframework.cloud:spring-cloud-context:2.2.2.RELEASE (*)
| | +--- org.springframework:spring-web:5.2.4.RELEASE -> 5.2.3.RELEASE (*)
| | +--- com.fasterxml.jackson.core:jackson-annotations:2.10.2
| | \--- com.fasterxml.jackson.core:jackson-databind:2.10.2 (*)
| \--- com.fasterxml.jackson.core:jackson-databind:2.10.2 (*)
+--- org.springframework.cloud:spring-cloud-config-server:2.2.2.RELEASE
| +--- org.springframework.cloud:spring-cloud-config-client:2.2.2.RELEASE (*)
| +--- org.springframework.boot:spring-boot-starter-actuator:2.2.5.RELEASE -> 2.2.4.RELEASE (*)
| +--- org.springframework.boot:spring-boot-starter-web:2.2.5.RELEASE -> 2.2.4.RELEASE
| | +--- org.springframework.boot:spring-boot-starter:2.2.4.RELEASE (*)
| | +--- org.springframework.boot:spring-boot-starter-json:2.2.4.RELEASE (*)
| | +--- org.springframework.boot:spring-boot-starter-tomcat:2.2.4.RELEASE
| | | +--- jakarta.annotation:jakarta.annotation-api:1.3.5
| | | +--- org.apache.tomcat.embed:tomcat-embed-core:9.0.30
| | | +--- org.apache.tomcat.embed:tomcat-embed-el:9.0.30
| | | \--- org.apache.tomcat.embed:tomcat-embed-websocket:9.0.30
| | | \--- org.apache.tomcat.embed:tomcat-embed-core:9.0.30
| | +--- org.springframework.boot:spring-boot-starter-validation:2.2.4.RELEASE (*)
| | +--- org.springframework:spring-web:5.2.3.RELEASE (*)
| | \--- org.springframework:spring-webmvc:5.2.3.RELEASE
| | +--- org.springframework:spring-aop:5.2.3.RELEASE (*)
| | +--- org.springframework:spring-beans:5.2.3.RELEASE (*)
| | +--- org.springframework:spring-context:5.2.3.RELEASE (*)
| | +--- org.springframework:spring-core:5.2.3.RELEASE (*)
| | +--- org.springframework:spring-expression:5.2.3.RELEASE (*)
| | \--- org.springframework:spring-web:5.2.3.RELEASE (*)
| +--- org.springframework.boot:spring-boot-starter-validation:2.2.5.RELEASE -> 2.2.4.RELEASE (*)
| +--- org.springframework.security:spring-security-crypto:5.2.2.RELEASE -> 5.2.1.RELEASE
| +--- org.springframework.security:spring-security-rsa:1.0.9.RELEASE (*)
| +--- org.eclipse.jgit:org.eclipse.jgit:5.1.3.201810200350-r
| | +--- com.jcraft:jsch:0.1.54
| | +--- com.jcraft:jzlib:1.1.1
| | +--- com.googlecode.javaewah:JavaEWAH:1.1.6
| | \--- org.slf4j:slf4j-api:1.7.2 -> 1.7.30
| +--- org.eclipse.jgit:org.eclipse.jgit.http.apache:5.1.3.201810200350-r
| | +--- org.eclipse.jgit:org.eclipse.jgit:5.1.3.201810200350-r (*)
| | +--- org.apache.httpcomponents:httpclient:4.5.5 -> 4.5.10
| | | +--- org.apache.httpcomponents:httpcore:4.4.12 -> 4.4.13
| | | +--- commons-logging:commons-logging:1.2
| | | \--- commons-codec:commons-codec:1.11 -> 1.13
| | \--- org.apache.httpcomponents:httpcore:4.4.9 -> 4.4.13
| +--- org.yaml:snakeyaml:1.25
| \--- com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.10.2
| +--- org.yaml:snakeyaml:1.24 -> 1.25
| \--- com.fasterxml.jackson.core:jackson-core:2.10.2
+--- org.springframework.cloud:spring-cloud-starter-openfeign:2.2.2.RELEASE
| +--- org.springframework.cloud:spring-cloud-starter:2.2.2.RELEASE (*)
| +--- org.springframework.cloud:spring-cloud-openfeign-core:2.2.2.RELEASE
| | +--- org.springframework.boot:spring-boot-autoconfigure:2.2.5.RELEASE -> 2.2.4.RELEASE (*)
| | +--- org.springframework.cloud:spring-cloud-netflix-ribbon:2.2.2.RELEASE
| | | \--- org.springframework.cloud:spring-cloud-netflix-archaius:2.2.2.RELEASE
| | +--- org.springframework.boot:spring-boot-starter-aop:2.2.5.RELEASE -> 2.2.4.RELEASE
| | | +--- org.springframework.boot:spring-boot-starter:2.2.4.RELEASE (*)
| | | +--- org.springframework:spring-aop:5.2.3.RELEASE (*)
| | | \--- org.aspectj:aspectjweaver:1.9.5
| | \--- io.github.openfeign.form:feign-form-spring:3.8.0
| | +--- io.github.openfeign.form:feign-form:3.8.0
| | | \--- org.slf4j:slf4j-api:1.7.26 -> 1.7.30
| | +--- org.springframework:spring-web:5.1.5.RELEASE -> 5.2.3.RELEASE (*)
| | +--- commons-fileupload:commons-fileupload:1.4
| | | \--- commons-io:commons-io:2.2 -> 2.5
| | \--- org.slf4j:slf4j-api:1.7.26 -> 1.7.30
| +--- org.springframework:spring-web:5.2.4.RELEASE -> 5.2.3.RELEASE (*)
| +--- org.springframework.cloud:spring-cloud-commons:2.2.2.RELEASE (*)
| +--- io.github.openfeign:feign-core:10.7.4
| +--- io.github.openfeign:feign-slf4j:10.7.4
| | +--- io.github.openfeign:feign-core:10.7.4
| | \--- org.slf4j:slf4j-api:1.7.13 -> 1.7.30
| \--- io.github.openfeign:feign-hystrix:10.7.4
| +--- io.github.openfeign:feign-core:10.7.4
| +--- com.netflix.archaius:archaius-core:0.7.6
| | +--- com.google.code.findbugs:jsr305:3.0.1 -> 3.0.2
| | +--- commons-configuration:commons-configuration:1.8
| | | +--- commons-lang:commons-lang:2.6
| | | \--- commons-logging:commons-logging:1.1.1 -> 1.2
| | +--- org.slf4j:slf4j-api:1.6.4 -> 1.7.30
| | +--- com.google.guava:guava:16.0 -> 28.2-android (*)
| | +--- com.fasterxml.jackson.core:jackson-annotations:2.4.3 -> 2.10.2
| | +--- com.fasterxml.jackson.core:jackson-core:2.4.3 -> 2.10.2
| | \--- com.fasterxml.jackson.core:jackson-databind:2.4.3 -> 2.10.2 (*)
| \--- com.netflix.hystrix:hystrix-core:1.5.18
| +--- org.slf4j:slf4j-api:1.7.0 -> 1.7.30
| +--- com.netflix.archaius:archaius-core:0.4.1 -> 0.7.6 (*)
| +--- io.reactivex:rxjava:1.2.0 -> 1.3.8
| \--- org.hdrhistogram:HdrHistogram:2.1.9 -> 2.1.11
+--- io.springfox:springfox-swagger2:3.0.0-SNAPSHOT (*)
+--- io.springfox:springfox-spring-webflux:3.0.0-SNAPSHOT (*)
+--- io.springfox:springfox-swagger-ui:3.0.0-SNAPSHOT
+--- javax.persistence:persistence-api:1.0.2
+--- io.projectreactor:reactor-core:3.3.3.RELEASE (*)
+--- com.microsoft.azure:applicationinsights-spring-boot-starter:2.5.1 (*)
\--- com.squareup.okhttp3:okhttp:3.12.2 (*)
Issue Analytics
- State:
- Created 4 years ago
- Comments:14 (10 by maintainers)
It will take some time to test this since we’ve switched over to gateway mode.
@timwhit yes, there will be.