TransferManager.DownloadAsync: blob type BlockBlob is not supported

Which service(blob, file) does this issue concern?

BlockBlob

Which version of the SDK was used?

Microsoft.Azure.Storage.DataMovement 1.0.0 Microsoft.Azure.Storage.Blob 10.0.3 Microsoft Azure Storage Emulator 5.9.0.0

On which platform were you using? (.Net Framework version or .Net Core version, and OS version)

.NET Core 2.1 on Windows 10

How can the problem be reproduced? It’d be better if the code caused the problem can be shared.

Trying to download a BlockBlob content to a MemoryStream via TransferManager.DownloadAsync:

var blobStorageAccount = CloudStorageAccount.Parse("UseDevelopmentStorage=true");
var blobClient = blobStorageAccount.CreateCloudBlobClient();
_blobContainer = blobClient.GetContainerReference("dev");

// using GetBlobBlockReference instead works
var blob = _blobContainer.GetBlobReference("test.jpg");
if (!await blob.ExistsAsync())
	return;	

var ms = new MemoryStream();
await TransferManager.DownloadAsync(blob, ms);

What problem was encountered?

DownloadAsync throws an exception saying that blob type BlockBlob is not supported:

InvalidOperationException: The given blob type BlockBlob is not supported.
Microsoft.Azure.Storage.DataMovement.TransferControllers.SyncTransferController.GetReader(TransferLocation sourceLocation) in SyncTransferController.cs
Microsoft.Azure.Storage.DataMovement.TransferControllers.SyncTransferController..ctor(TransferScheduler transferScheduler, TransferJob transferJob, CancellationToken userCancellationToken) in SyncTransferController.cs
Microsoft.Azure.Storage.DataMovement.TransferScheduler.GenerateTransferConstroller(TransferScheduler transferScheduler, TransferJob transferJob, CancellationToken cancellationToken) in TransferScheduler.cs
Microsoft.Azure.Storage.DataMovement.TransferScheduler.ExecuteJobInternalAsync(TransferJob job, CancellationToken cancellationToken) in TransferScheduler.cs
Microsoft.Azure.Storage.DataMovement.SingleObjectTransfer.ExecuteAsync(TransferScheduler scheduler, CancellationToken cancellationToken) in SingleObjectTransfer.cs
Microsoft.Azure.Storage.DataMovement.TransferManager.DoTransfer(Transfer transfer, TransferContext transferContext, CancellationToken cancellationToken) in TransferManager.cs
MyCompany.DocumentService.BlobService.DownloadFile(string path) in BlobService.cs
+
			await TransferManager.DownloadAsync(blob, ms);

Have you found a mitigation/solution?

If I get a direct reference to the BlockBlob via _blobContainer.GetBlockBlobReference, it works as expected, but I find it confusing because I can use CloudBlob.DownloadToStreamAsync instead, and I don’t need to know the underlying BlobType.

[blueww]

@metoule Thanks for the change suggestion of error message! We will consider to fix it in a future release.

[blueww]

@metoule

This issue is almost same as https://github.com/Azure/azure-storage-net-data-movement/issues/33

To accelerate the transfer, DMlib use different APIs related to different BlobTypes to do Blob transfer, so the blobType need to be defined before input the blob object to DMlib.

And since CloudBlob doesn’t have the blobtype related APIs, but only CloudBlockBlob,CloudPageBlob,CloudAppendBlob have, so we Need these Type Object to run the blob transfer. And We can’t convert CloudBlob directly to these type object in DMlib code, since some blob property might missing in the convert and take bugs.

For your scenario, we would suggest you create CloubBlockBlob object to run download.