question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

[BUG] OutOfMemoryError when calling ShareFileAsyncClient.getProperties() after `azure-storage-file-share` lib upgrade

See original GitHub issue

Describe the bug When using azure-storage-file-share version 12.15.1 and 12.16.0 I got an OutOfMemoryError when downloading large file (325M) with ShareFileAsyncClient.downloadWithResponse() No such error with version 12.14.0

Exception or Stack Trace

java.lang.OutOfMemoryError: Java heap space
	at com.azure.core.implementation.ByteBufferCollector.<init>(ByteBufferCollector.java:52) ~[azure-core-1.33.0.jar:1.33.0]
	at com.azure.core.util.FluxUtil.lambda$collectBytesInByteBufferStream$2(FluxUtil.java:139) ~[azure-core-1.33.0.jar:1.33.0]
	at com.azure.core.util.FluxUtil$$Lambda$1447/0x00000008014b4fc0.get(Unknown Source) ~[?:?]
	at reactor.core.publisher.MonoCollect.subscribeOrReturn(MonoCollect.java:58) ~[reactor-core-3.4.24.jar:3.4.24]
	at reactor.core.publisher.Flux.subscribe(Flux.java:8512) ~[reactor-core-3.4.24.jar:3.4.24]
	at reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain.onNext(MonoFlatMapMany.java:195) ~[reactor-core-3.4.24.jar:3.4.24]
	at reactor.core.publisher.SerializedSubscriber.onNext(SerializedSubscriber.java:99) ~[reactor-core-3.4.24.jar:3.4.24]
	at reactor.core.publisher.FluxRetryWhen$RetryWhenMainSubscriber.onNext(FluxRetryWhen.java:174) ~[reactor-core-3.4.24.jar:3.4.24]
	at reactor.core.publisher.MonoCreate$DefaultMonoSink.success(MonoCreate.java:172) ~[reactor-core-3.4.24.jar:3.4.24]
	at reactor.netty.http.client.HttpClientConnect$HttpIOHandlerObserver.onStateChange(HttpClientConnect.java:431) ~[reactor-netty-http-1.0.24.jar:1.0.24]
	at reactor.netty.ReactorNetty$CompositeConnectionObserver.onStateChange(ReactorNetty.java:677) ~[reactor-netty-core-1.0.24.jar:1.0.24]
	at reactor.netty.resources.DefaultPooledConnectionProvider$DisposableAcquire.onStateChange(DefaultPooledConnectionProvider.java:187) ~[reactor-netty-core-1.0.24.jar:1.0.24]
	at reactor.netty.resources.DefaultPooledConnectionProvider$PooledConnection.onStateChange(DefaultPooledConnectionProvider.java:444) ~[reactor-netty-core-1.0.24.jar:1.0.24]
	at reactor.netty.http.client.HttpClientOperations.onInboundNext(HttpClientOperations.java:637) ~[reactor-netty-http-1.0.24.jar:1.0.24]
	at reactor.netty.channel.ChannelOperationsHandler.channelRead(ChannelOperationsHandler.java:113) ~[reactor-netty-core-1.0.24.jar:1.0.24]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) ~[netty-transport-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[netty-transport-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[netty-transport-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436) ~[netty-transport-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:336) ~[netty-codec-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:308) ~[netty-codec-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251) ~[netty-transport-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) ~[netty-transport-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[netty-transport-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[netty-transport-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1373) ~[netty-handler-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1236) ~[netty-handler-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1285) ~[netty-handler-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:519) ~[netty-codec-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:458) ~[netty-codec-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:280) ~[netty-codec-4.1.84.Final.jar:4.1.84.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) ~[netty-transport-4.1.84.Final.jar:4.1.84.Final]

To Reproduce Limit your heap size, and download a large file with code below. Jvm options used: -Xms64M -Xmx256M -Xss256k -XX:+UseStringDeduplication -XX:MaxHeapFreeRatio=30 -XX:MinHeapFreeRatio=10

Code Snippet

        final ShareFileAsyncClient shareFileClient = ...
        final StreamingResponseBody streamingResponseBody = outputStream ->
                shareFileClient.downloadWithResponse(fileRange, null)
                        .flatMap(response -> FluxUtil.writeToOutputStream(response.getValue(), outputStream)).block();
        return responseEntity
                .cacheControl(CacheControl.maxAge(Duration.ofDays(10L)))
                .contentType(VIDEO_MP4)
                .contentLength(contentLength)
                .eTag(etag)
                .lastModified(lastModifiedInstant)
                .body(streamingResponseBody);

Expected behavior No OutOfMemoryError.

Setup (please complete the following information):

  • OS: Windows 10
  • IDE: IntelliJ
  • Library/Libraries: com.azure:azure-storage-file-share:12.16.0
  • Java version: OpenJDK Runtime Environment Temurin-17.0.5+8 (build 17.0.5+8)
  • App Server/Environment: Tomcat
  • Frameworks: Spring-boot v2.7.5

Information Checklist Kindly make sure that you have added all the following information above and checkoff the required fields otherwise we will treat the issuer as an incomplete report

  • Bug Description Added
  • Repro Steps Added
  • Setup information Added

Issue Analytics

  • State:closed
  • Created a year ago
  • Comments:5 (3 by maintainers)

github_iconTop GitHub Comments

1reaction
alzimmermsftcommented, Nov 1, 2022

@ibrahimrabab this looks similar to #31555. Both issues appear to be happening after the July 2022 releases which helps scope this to a change made between July and August.

0reactions
croudetcommented, Nov 1, 2022

We are using connection string.

While debugging I saw a call to public ByteBufferCollector(int initialSize) with initialSize of 340097844, which is the size of the file.

Read more comments on GitHub >

github_iconTop Results From Across the Web

[BUG] Java Heap OutOfMemoryError after library upgrade
Describe the bug Hello, dear Azure Team! We have faced an interesting memory issue when upgrading the storage blob library.
Read more >
How to deal with "java.lang.OutOfMemoryError: Java heap ...
Recently, I am running into java.lang.OutOfMemoryError: Java heap space error because I am not being conservative on memory usage. The user can open...
Read more >
3.2 Understand the OutOfMemoryError Exception
OutOfMemoryError exception. Usually, this error is thrown when there is insufficient space to allocate an object in the Java heap. In this case,...
Read more >
How to Handle the OutOfMemoryError in Java - Rollbar
A java.lang.OutOfMemoryError is a runtime error in Java which occurs when the JVM is unable to allocate an object due to insufficient space ......
Read more >
How to fix out of memory errors by increasing available memory
Step 1: Increase Available Memory · Increase Xmx in small increments (eg 512mb at a time), until you no longer experience the OutOfMemory...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found