[BUG] Storage: Container creation fails due to header format

Library name and version

Azure.Storage.Blobs (nightly build)

Describe the bug

When attempting to create a container using BlobContainerClient::CreateAsync, the call is failing with an HTTP 400 and a message referencing that the value of the x-ms-version header is not in the correct format.

This was observed during test runs, starting last week in Canary and only appears to impact legs with project references. At present, the Public cloud is unaffected, as are package references to v12.12.0.

Expected behavior

The container is created without an error.

Actual behavior

Error details:

Azure.RequestFailedException : The value for one of the HTTP headers is not in the correct format.
RequestId:64fefab0-701e-0041-6e2f-77aec5000000
Time:2022-06-03T09:48:04.2197881Z
Status: 400 (The value for one of the HTTP headers is not in the correct format.)
ErrorCode: InvalidHeaderValue

Additional Information:
HeaderName: x-ms-version
HeaderValue: 2021-08-06

Content:

<Error>InvalidHeaderValue<Message>The value for one of the HTTP headers is not in the correct format.
RequestId:64fefab0-701e-0041-6e2f-77aec5000000
Time:2022-06-03T09:48:04.2197881Z</Message><HeaderName>x-ms-version</HeaderName><HeaderValue>2021-08-06</HeaderValue></Error>

Headers:
Server: Windows-Azure-Blob/1.0,Microsoft-HTTPAPI/2.0
x-ms-request-id: 64fefab0-701e-0041-6e2f-77aec5000000
Date: Fri, 03 Jun 2022 09:48:03 GMT
Content-Length: 329
Content-Type: application/xml

Data:
HeaderName: x-ms-version
HeaderValue: 2021-08-06

Stack trace:

at Azure.Storage.Blobs.ContainerRestClient.CreateAsync(Nullable`1 timeout, IDictionary`2 metadata, Nullable`1 access, String defaultEncryptionScope, Nullable`1 preventEncryptionScopeOverride, CancellationToken cancellationToken) in /Users/runner/work/1/s/sdk/storage/Azure.Storage.Blobs/src/Generated/ContainerRestClient.cs:line 94
   at Azure.Storage.Blobs.BlobContainerClient.CreateInternal(PublicAccessType publicAccessType, IDictionary`2 metadata, BlobContainerEncryptionScopeOptions encryptionScopeOptions, Boolean async, CancellationToken cancellationToken, String operationName) in /Users/runner/work/1/s/sdk/storage/Azure.Storage.Blobs/src/BlobContainerClient.cs:line 1166
   at Azure.Storage.Blobs.BlobContainerClient.CreateAsync(PublicAccessType publicAccessType, IDictionary`2 metadata, BlobContainerEncryptionScopeOptions encryptionScopeOptions, CancellationToken cancellationToken) in /Users/runner/work/1/s/sdk/storage/Azure.Storage.Blobs/src/BlobContainerClient.cs:line 728
   at Azure.Messaging.EventHubs.Tests.StorageScope.CreateAsync(String caller) in /Users/runner/work/1/s/sdk/eventhub/Azure.Messaging.EventHubs.Shared/src/BlobStorageTesting/StorageScope.cs:line 103
   at Azure.Messaging.EventHubs.Tests.StorageScope.CreateAsync(String caller) in /Users/runner/work/1/s/sdk/eventhub/Azure.Messaging.EventHubs.Shared/src/BlobStorageTesting/StorageScope.cs:line 119
   at Azure.Messaging.EventHubs.Tests.Snippets.Sample05_IdentityAndSharedAccessCredentialsLiveTests.DefaultAzureCredential() in /Users/runner/work/1/s/sdk/eventhub/Azure.Messaging.EventHubs.Processor/tests/Snippets/Sample05_IdentityAndSharedAccessCredentialsLiveTests.cs:line 35
   at Azure.Messaging.EventHubs.Tests.Snippets.Sample05_IdentityAndSharedAccessCredentialsLiveTests.DefaultAzureCredential() in /Users/runner/work/1/s/sdk/eventhub/Azure.Messaging.EventHubs.Processor/tests/Snippets/Sample05_IdentityAndSharedAccessCredentialsLiveTests.cs:line 114
   at NUnit.Framework.Internal.TaskAwaitAdapter.GenericAdapter`1.BlockUntilCompleted()
   at NUnit.Framework.Internal.MessagePumpStrategy.NoMessagePumpStrategy.WaitForCompletion(AwaitAdapter awaiter)
   at NUnit.Framework.Internal.AsyncToSyncAdapter.Await(Func`1 invoke)
   at NUnit.Framework.Internal.Commands.TestMethodCommand.RunTestMethod(TestExecutionContext context)
   at NUnit.Framework.Internal.Commands.TestMethodCommand.Execute(TestExecutionContext context)
   at NUnit.Framework.Internal.Execution.SimpleWorkItem.<>c__DisplayClass4_0.<PerformWork>b__0()
   at NUnit.Framework.Internal.ContextUtils.<>c__DisplayClass1_0`1.<DoIsolated>b__0(Object _)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location ---
   at NUnit.Framework.Internal.ContextUtils.DoIsolated(ContextCallback callback, Object state)
   at NUnit.Framework.Internal.ContextUtils.DoIsolated[T](Func`1 func)
   at NUnit.Framework.Internal.Execution.SimpleWorkItem.PerformWork()

More context can be found in the Event Hubs live test run from Friday, June 3. (Microsoft Internal)

Reproduction Steps

Sample code:

var containerName = $"{ Guid.NewGuid().ToString("D").Substring(0, 13) }-{ caller }";
var containerClient = new BlobContainerClient(StorageTestEnvironment.Instance.StorageConnectionString, containerName);

await containerClient.CreateAsync().ConfigureAwait(false);

Full source can be found in StorageScope, part of the Event Hubs test infrastructure.

Environment

Azure SDK DevOps pipelines; live test runs.

[seanmcc-msft]

Hi @jsquire, I have also been experiencing this issue. This is being caused by the latest version, 2021-08-06 having been rolled out in production, but not canary. It is not an issue with the SDK. I can work with the backend teams to get this version enabled in canary.

[jsquire]

This appears to be resolved; closing.