[BUG] Can't set PrecalculatedChecksum - "Precalculated checksum not supported when potentially partitioning an upload."

Library name and version

Azure.Storage.Blobs 12.16.0

Describe the bug

Could be me missing important bit in documentation, but I searched docs, stackoverflow and source code and I can’t see the solution. Decided to start as bug report.

Goal: upload block blob from stream, with purposefully incorrect precalculated MD5, and get expected error that hash is incorrect.

Looking at the code, we throw this error before attempting to determine if the blob can be uploaded in one operation (and supplying checksum would actually be valid).

Expected behavior

Failure to upload blob due to incorrect MD5

Actual behavior

System.ArgumentException HResult=0x80070057 Message=Precalculated checksum not supported when potentially partitioning an upload. Source=Azure.Storage.Blobs StackTrace: at Azure.Storage.PartitionedUploader2..ctor(Behaviors behaviors, StorageTransferOptions transferOptions, UploadTransferValidationOptions transferValidation, ArrayPool1 arrayPool, String operationName) at Azure.Storage.Blobs.Specialized.BlockBlobClient.GetPartitionedUploader(StorageTransferOptions transferOptions, UploadTransferValidationOptions validationOptions, ArrayPool`1 arrayPool, String operationName) at Azure.Storage.Blobs.Specialized.BlockBlobClient.<UploadAsync>d__30.MoveNext() <user code>

Reproduction Steps

            BlobContainerClient containerClient = new BlobContainerClient(new Uri("https://127.0.0.1:10000/devstoreaccount1/container-name"), new DefaultAzureCredential());

            BlockBlobClient blobClient = containerClient.GetBlockBlobClient("test.bin");

            BlobUploadOptions options = new BlobUploadOptions()
            {
                TransferValidation = new UploadTransferValidationOptions() { ChecksumAlgorithm = StorageChecksumAlgorithm.MD5, PrecalculatedChecksum = Convert.FromBase64String("BadChecksum="), },
                //TransferOptions = new StorageTransferOptions() { MaximumConcurrency = 1 },
            };

            MemoryStream stream = new System.IO.MemoryStream(Encoding.UTF8.GetBytes("Hello, World!"));
            await blobClient.UploadAsync(stream, options, cancellationToken: CancellationToken.None); // <-- exception here

Environment

Win 11, 8.0.100-preview.3.23178.7, VS 2022 (Version 17.6.0 Preview 4.0)

[jaschrep-msft]

Looking at the docs for BlobUploadOptions.TransferValidation:

This operation does not allow PrecalculatedChecksum to be set.

Additionally, the docs for the precalculated checksum property state:

Not all upload APIs can use this value, and will throw if one is provided. Please check documentation on specific APIs for whether this can be used.

@ayeltsov I think these docs do explain the trouble you were having, and they appear to be unchanged across versions since their introduction. However, it does look like the latest version should be updated to state that the property can be used if using CRC. I’ve filed #37882 to update that. Otherwise, I think this issue is resolved.

[ayeltsov]

Confirmed this is working on 12.17. In P.S. there’s a sample in case someone stumbles over this thread.

@jaschrep-msft - feel free to resolve but maybe check if any doc follow-up tasks are needed and worth investment. I was trying to find my way with documentation and it was challenging.

P.S. This code follows @jaschrep-msft 's suggestion:

            // Did not work with emulator
            BlobContainerClient containerClient = await GetRealBlobContainerClient();

            BlobClient blobClient = containerClient.GetBlobClient(blobName: "test.txt");

            byte[] content = "Hello, World!"u8.ToArray();

            StorageCrc64HashAlgorithm crc64 = StorageCrc64HashAlgorithm.Create();
            crc64.Append(content);

           BlobUploadOptions options = new()
            {
                TransferValidation = new UploadTransferValidationOptions()
                {
                    ChecksumAlgorithm = StorageChecksumAlgorithm.StorageCrc64,
                    PrecalculatedChecksum = crc64.GetCurrentHash(),
                },
            };

            blobClient.Upload(new MemoryStream(content), options);

And here’s expected exception when I supply incorrect CRC:

Unhandled exception. Azure.RequestFailedException: The CRC64 value specified in the request did not match with the CRC64 value calculated by the server.
RequestId:1ae17ec7-501e-0036-206f-beb9b1000000
Time:2023-07-24T20:44:26.7474653Z
Status: 400 (The CRC64 value specified in the request did not match with the CRC64 value calculated by the server.)
ErrorCode: Crc64Mismatch

Additional Information:
UserSpecifiedCrc: peH8Xsgc5QI=
ServerCalculatedCrc: TdKtJkO+qdQ=
<...>