[BUG] OutOfMemoryError when calling ShareFileAsyncClient.getProperties() after `azure-storage-file-share` lib upgrade
See original GitHub issueDescribe the bug
When using azure-storage-file-share
version 12.15.1
and 12.16.0
I got an OutOfMemoryError
when downloading large file (325M) with ShareFileAsyncClient.downloadWithResponse()
No such error with version 12.14.0
Exception or Stack Trace
java.lang.OutOfMemoryError: Java heap space
at com.azure.core.implementation.ByteBufferCollector.<init>(ByteBufferCollector.java:52) ~[azure-core-1.33.0.jar:1.33.0]
at com.azure.core.util.FluxUtil.lambda$collectBytesInByteBufferStream$2(FluxUtil.java:139) ~[azure-core-1.33.0.jar:1.33.0]
at com.azure.core.util.FluxUtil$$Lambda$1447/0x00000008014b4fc0.get(Unknown Source) ~[?:?]
at reactor.core.publisher.MonoCollect.subscribeOrReturn(MonoCollect.java:58) ~[reactor-core-3.4.24.jar:3.4.24]
at reactor.core.publisher.Flux.subscribe(Flux.java:8512) ~[reactor-core-3.4.24.jar:3.4.24]
at reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain.onNext(MonoFlatMapMany.java:195) ~[reactor-core-3.4.24.jar:3.4.24]
at reactor.core.publisher.SerializedSubscriber.onNext(SerializedSubscriber.java:99) ~[reactor-core-3.4.24.jar:3.4.24]
at reactor.core.publisher.FluxRetryWhen$RetryWhenMainSubscriber.onNext(FluxRetryWhen.java:174) ~[reactor-core-3.4.24.jar:3.4.24]
at reactor.core.publisher.MonoCreate$DefaultMonoSink.success(MonoCreate.java:172) ~[reactor-core-3.4.24.jar:3.4.24]
at reactor.netty.http.client.HttpClientConnect$HttpIOHandlerObserver.onStateChange(HttpClientConnect.java:431) ~[reactor-netty-http-1.0.24.jar:1.0.24]
at reactor.netty.ReactorNetty$CompositeConnectionObserver.onStateChange(ReactorNetty.java:677) ~[reactor-netty-core-1.0.24.jar:1.0.24]
at reactor.netty.resources.DefaultPooledConnectionProvider$DisposableAcquire.onStateChange(DefaultPooledConnectionProvider.java:187) ~[reactor-netty-core-1.0.24.jar:1.0.24]
at reactor.netty.resources.DefaultPooledConnectionProvider$PooledConnection.onStateChange(DefaultPooledConnectionProvider.java:444) ~[reactor-netty-core-1.0.24.jar:1.0.24]
at reactor.netty.http.client.HttpClientOperations.onInboundNext(HttpClientOperations.java:637) ~[reactor-netty-http-1.0.24.jar:1.0.24]
at reactor.netty.channel.ChannelOperationsHandler.channelRead(ChannelOperationsHandler.java:113) ~[reactor-netty-core-1.0.24.jar:1.0.24]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) ~[netty-transport-4.1.84.Final.jar:4.1.84.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[netty-transport-4.1.84.Final.jar:4.1.84.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[netty-transport-4.1.84.Final.jar:4.1.84.Final]
at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436) ~[netty-transport-4.1.84.Final.jar:4.1.84.Final]
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:336) ~[netty-codec-4.1.84.Final.jar:4.1.84.Final]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:308) ~[netty-codec-4.1.84.Final.jar:4.1.84.Final]
at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251) ~[netty-transport-4.1.84.Final.jar:4.1.84.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) ~[netty-transport-4.1.84.Final.jar:4.1.84.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[netty-transport-4.1.84.Final.jar:4.1.84.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[netty-transport-4.1.84.Final.jar:4.1.84.Final]
at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1373) ~[netty-handler-4.1.84.Final.jar:4.1.84.Final]
at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1236) ~[netty-handler-4.1.84.Final.jar:4.1.84.Final]
at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1285) ~[netty-handler-4.1.84.Final.jar:4.1.84.Final]
at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:519) ~[netty-codec-4.1.84.Final.jar:4.1.84.Final]
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:458) ~[netty-codec-4.1.84.Final.jar:4.1.84.Final]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:280) ~[netty-codec-4.1.84.Final.jar:4.1.84.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) ~[netty-transport-4.1.84.Final.jar:4.1.84.Final]
To Reproduce
Limit your heap size, and download a large file with code below.
Jvm options used: -Xms64M -Xmx256M -Xss256k -XX:+UseStringDeduplication -XX:MaxHeapFreeRatio=30 -XX:MinHeapFreeRatio=10
Code Snippet
final ShareFileAsyncClient shareFileClient = ...
final StreamingResponseBody streamingResponseBody = outputStream ->
shareFileClient.downloadWithResponse(fileRange, null)
.flatMap(response -> FluxUtil.writeToOutputStream(response.getValue(), outputStream)).block();
return responseEntity
.cacheControl(CacheControl.maxAge(Duration.ofDays(10L)))
.contentType(VIDEO_MP4)
.contentLength(contentLength)
.eTag(etag)
.lastModified(lastModifiedInstant)
.body(streamingResponseBody);
Expected behavior
No OutOfMemoryError
.
Setup (please complete the following information):
- OS: Windows 10
- IDE: IntelliJ
- Library/Libraries:
com.azure:azure-storage-file-share:12.16.0
- Java version:
OpenJDK Runtime Environment Temurin-17.0.5+8 (build 17.0.5+8)
- App Server/Environment: Tomcat
- Frameworks: Spring-boot v2.7.5
Information Checklist Kindly make sure that you have added all the following information above and checkoff the required fields otherwise we will treat the issuer as an incomplete report
- Bug Description Added
- Repro Steps Added
- Setup information Added
Issue Analytics
- State:
- Created a year ago
- Comments:5 (3 by maintainers)
Top Results From Across the Web
[BUG] Java Heap OutOfMemoryError after library upgrade
Describe the bug Hello, dear Azure Team! We have faced an interesting memory issue when upgrading the storage blob library.
Read more >How to deal with "java.lang.OutOfMemoryError: Java heap ...
Recently, I am running into java.lang.OutOfMemoryError: Java heap space error because I am not being conservative on memory usage. The user can open...
Read more >3.2 Understand the OutOfMemoryError Exception
OutOfMemoryError exception. Usually, this error is thrown when there is insufficient space to allocate an object in the Java heap. In this case,...
Read more >How to Handle the OutOfMemoryError in Java - Rollbar
A java.lang.OutOfMemoryError is a runtime error in Java which occurs when the JVM is unable to allocate an object due to insufficient space ......
Read more >How to fix out of memory errors by increasing available memory
Step 1: Increase Available Memory · Increase Xmx in small increments (eg 512mb at a time), until you no longer experience the OutOfMemory...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
@ibrahimrabab this looks similar to #31555. Both issues appear to be happening after the July 2022 releases which helps scope this to a change made between July and August.
We are using connection string.
While debugging I saw a call to
public ByteBufferCollector(int initialSize)
with initialSize of 340097844, which is the size of the file.