StorageException in DownloadToByteArrayAsync

I use DownloadToByteArrayAsync to read from the end of an Append Blob, using this call:

var position = ...; // The value of 'blob.Properties.Length' at some point in the past

var context = new OperationContext();
context.SendingRequest += (sender, e) => { e.Request.Headers["if-match"] = "*"; };

var length = await blob.DownloadRangeToByteArrayAsync(
  _buffer,
  0,
  position, 
  _buffer.Length,
  new AccessCondition(),
  new BlobRequestOptions(),
  context,
  cancel);

I expect this call to read any bytes past position into _buffer, and to return the number of bytes read. However, if another process is currently appending to the blob, this call sometimes throws an exception:

Microsoft.WindowsAzure.Storage.StorageException: Incorrect number of bytes received. Expected '3443580', received '3443752'
at Microsoft.WindowsAzure.Storage.Core.Util.StorageAsyncResult`1.End()
at Microsoft.WindowsAzure.Storage.Blob.CloudBlob.EndDownloadRangeToByteArray(System.IAsyncResult)
at Microsoft.WindowsAzure.Storage.Core.Util.AsyncExtensions+<>c__DisplayClass1`1.<CreateCallback>b__0(System.IAsyncResult)
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(System.Threading.Tasks.Task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(System.Threading.Tasks.Task)
at AzureEventStore.Core.Drivers.AzureStorageDriver+<ReadAsync>d__15.MoveNext()

[VictorNicollet]

Thank you. Again, I wouldn’t really expect this to work on an arbitrary Block Blob (because the contents can change at random), but I do expect an Append blob to be append-only.

[VictorNicollet]

It was my understanding (but the documentation is vague on this point) that the DownloadRangeAsByteArrayAsync functions returned the number of bytes read because that number could be smaller than the requested length (in cases where offset + length exceeds the length of the blob). If the expected behavior is that the call should fail if the range goes past the end of the blob (and therefore, the return value is always the length parameter), then I should probably look for another solution.

My application reads from an Append Blob, polling for new data every few seconds (this new data is appended by other processes). I perform this call as part of the polling loop, each time with a position equal to the length of the blob and a _buffer.Length of 4MB. The observed behavior is that:

• if no concurrent write happened since the last poll, it returns a length of 0 and reads no bytes.
• if a concurrent write happened, it returns a length no larger than 4MB (but often smaller, if only a little data was written). My code then adds this length to position and performs the call again.
• in rare cases, if a concurrent write has happened, the above StorageException is thrown, but most concurrent writes do not cause the exception.

So, it certainly looks like DownloadRangeAsByteArrayAsync behaves as I expect it to, outside of those rare exceptions being thrown, but I could be relying on unintentional (or unspecified) behavior.

I am unable to provide a value of position + _buffer.Length that is smaller than the length of the blob, because I do not have access to the length of the blob. I could query for the blob metadata, but this would require two requests instead of only one in a rather latency-sensitive loop, for what I believe is a fairly simple “get new data” operation, so I would rather craft the HTTP GET request with an x-ms-range manually if doing so is not possible through the C# API.

Could I perhaps assume that if I do not provide the length parameter to DownloadRangeAsByteArrayAsync, the function will not attempt to read more data than necessary to fill the provided buffer ?