question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

[FEATURE REQUEST] OpenRead(): loop downloads for Read() calls larger than internal buffer

See original GitHub issue

Describe the bug We discovered an issue that when we use OpenRead on BlockBlobClient, the returned stream doesn’t handle reads of more than 4mb well. When reading more than 4mb, it only reads 4mb:

image

Expected behavior I expect that when I read the entire length of the stream into a buffer, it reads the entire stream

Actual behavior (include Exception or Stack Trace) The current behavior seems to be that even when you pass a number larger than 4mb to the read method, it only reads 4mb.

To Reproduce Use the following snippet to download a file larger than 4mb from a blob container:

 var connectionString = "<connectionstring>";
 var blobContainerName = "<containername>";
 var blobName = "<blobName>";
 var targetPath = "<targetPath>";

 var blobClient = new BlobServiceClient(connectionString);
 var blobContainer = blobClient.GetBlobContainerClient(blobContainerName);
 var blob = blobContainer.GetBlockBlobClient(blobName);

 using (var stream = blob.OpenRead())
 {
    using (var destStream = new System.IO.FileStream(targetPath, System.IO.FileMode.Create, System.IO.FileAccess.ReadWrite))
    {
        byte[] buffer = new byte[stream.Length];
        var read = stream.Read(buffer, 0, (int)stream.Length);
        destStream.Write(buffer, 0, (int)stream.Length);
    }
}

Environment:

  • Name and version of the Library package used:Azure.Storage.Blobs 12.7.0

Issue Analytics

  • State:open
  • Created 3 years ago
  • Comments:16 (10 by maintainers)

github_iconTop GitHub Comments

1reaction
tg-msftcommented, Nov 18, 2021

Hi folks – I don’t think we should make this change for a couple of reasons:

  • We don’t like to redefine semantics for .NET types in Azure libraries. Stream.Read has behaved like this forever. Even FileStream.Read cited above does not always return the number of bytes requested (reference source).
  • If we were to implement this, I think it should be a separate BlockingRead method rather than a mode of the Stream. I’d rather be very explicit about APIs that could negatively impact performance. That idea should be filed in https://github.com/dotnet/runtime for the .NET team to evaluate, if desired.
  • The workaround of wrapping in a “blocking Stream” is straightforward and not too onerous when this behavior is required.
0reactions
jaschrep-msftcommented, Jan 20, 2021

Reopening as a feature request to loop current Read()/ReadAsync() implementation, rather than force the caller to do so.

Read more comments on GitHub >

github_iconTop Results From Across the Web

When do a C# Webclient.OpenRead actually download ...
It depends on the protocol, size of the download vs. buffer size and may even depend on the web server configuration.
Read more >
4. Advanced File I/O - Linux System Programming [Book]
Allows a single call to read or write data to and from many buffers at once; useful for bunching together fields of different...
Read more >
System.Buffers - .NET
This article provides an overview of types that help read data that runs across multiple buffers. They're primarily used to support ...
Read more >
System.IO.Pipelines in .NET
Parse all the lines returned in the buffer. It's possible that the line is bigger than 1 KB (1024 bytes). The code needs...
Read more >
Understanding Streams in Node.js - NodeSource
The read() function reads some data from the internal buffer and returns it. When there is nothing to read, it returns null. So,...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found