Investigate ByteBuf resource leak Azure-KeyValut

Steps to reproduce:

Set up KeyVault Client with azure-security-keyvault-secrets: 4.1.0 and override Netty from 0.9.0 to 0.9.4 (as done by SpringBoot dependencies management) Fetch a few entries from KeyVault

2020-01-24 15:41:04,606 [      ] [reactor-http-nio-1] ERROR io.netty.util.ResourceLeakDetector.reportTracedLeak(320) - LEAK: ByteBuf.release() was not called before it's garbage-collected. See https://netty.io/wiki/reference-counted-objects.html for more information.
Recent access records:
Created at:
	io.netty.buffer.PooledByteBufAllocator.newDirectBuffer(PooledByteBufAllocator.java:349)
	io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:187)
	io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:178)
	io.netty.handler.ssl.SslHandler.allocate(SslHandler.java:2134)
	io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1319)
	io.netty.handler.ssl.SslHandler.decodeNonJdkCompatible(SslHandler.java:1231)
	io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1268)
	io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:498)
	io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:437)
	io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
	io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
	io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
	io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
	io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
	io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
	io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
	io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
	io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
	io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
	io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	java.base/java.lang.Thread.run(Thread.java:834)

[anuchandy]

@allenhumphreys, @blacelle - I can repro this using the dependencies in the maven tree. Looking into this.

[anuchandy]

@tysonnorris I just want to provide bit more details on reason I requested a new issue.

The original problem specifically was due to key-vault library not closing stream in the pipeline.

The ByteBuf leak could happen if the stream is not closed in the library or in the user app. We need to investigate the issue you are seeing in the context of cosmos-sdk, I’m opening a new issue and will tag you in it.

Sorry that this issue’s title is misleading, I just corrected it to avoid any confusion in the future.