[FEATURE REQUEST] OpenRead(): loop downloads for Read() calls larger than internal buffer
See original GitHub issueDescribe the bug
We discovered an issue that when we use OpenRead
on BlockBlobClient
, the returned stream doesn’t handle reads of more than 4mb well. When reading more than 4mb, it only reads 4mb:
Expected behavior I expect that when I read the entire length of the stream into a buffer, it reads the entire stream
Actual behavior (include Exception or Stack Trace) The current behavior seems to be that even when you pass a number larger than 4mb to the read method, it only reads 4mb.
To Reproduce Use the following snippet to download a file larger than 4mb from a blob container:
var connectionString = "<connectionstring>";
var blobContainerName = "<containername>";
var blobName = "<blobName>";
var targetPath = "<targetPath>";
var blobClient = new BlobServiceClient(connectionString);
var blobContainer = blobClient.GetBlobContainerClient(blobContainerName);
var blob = blobContainer.GetBlockBlobClient(blobName);
using (var stream = blob.OpenRead())
{
using (var destStream = new System.IO.FileStream(targetPath, System.IO.FileMode.Create, System.IO.FileAccess.ReadWrite))
{
byte[] buffer = new byte[stream.Length];
var read = stream.Read(buffer, 0, (int)stream.Length);
destStream.Write(buffer, 0, (int)stream.Length);
}
}
Environment:
- Name and version of the Library package used:Azure.Storage.Blobs 12.7.0
Issue Analytics
- State:
- Created 3 years ago
- Comments:16 (10 by maintainers)
Top Results From Across the Web
When do a C# Webclient.OpenRead actually download ...
It depends on the protocol, size of the download vs. buffer size and may even depend on the web server configuration.
Read more >4. Advanced File I/O - Linux System Programming [Book]
Allows a single call to read or write data to and from many buffers at once; useful for bunching together fields of different...
Read more >System.Buffers - .NET
This article provides an overview of types that help read data that runs across multiple buffers. They're primarily used to support ...
Read more >System.IO.Pipelines in .NET
Parse all the lines returned in the buffer. It's possible that the line is bigger than 1 KB (1024 bytes). The code needs...
Read more >Understanding Streams in Node.js - NodeSource
The read() function reads some data from the internal buffer and returns it. When there is nothing to read, it returns null. So,...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
Hi folks – I don’t think we should make this change for a couple of reasons:
Stream.Read
has behaved like this forever. EvenFileStream.Read
cited above does not always return the number of bytes requested (reference source).BlockingRead
method rather than a mode of theStream
. I’d rather be very explicit about APIs that could negatively impact performance. That idea should be filed in https://github.com/dotnet/runtime for the .NET team to evaluate, if desired.Stream
” is straightforward and not too onerous when this behavior is required.Reopening as a feature request to loop current
Read()
/ReadAsync()
implementation, rather than force the caller to do so.