[BUG] Azure.Stoarge.Blobs 12.11.0 not working with azurite

Library name and version

Azure.Storage.Blobs

Describe the bug

When I upgraded from 12.10.0 to 12.11.0 our asp mvc core .net 6 application would no longer start. At the time the Azurite version was 3.14 and it was setup in visual studio per the instructions on this page. The error in this case occurred when the function:

await container.ExistsAsync()

This function is called as a part of our general azure file access which we call in startup.cs to get key files to setup dataprotection. The error was:

<Error> <Code>InvalidHeaderValue</Code> <Message>The value for one of the HTTP headers is not in the correct format. RequestId:d39d2cad-301e-009e-1546-3940de000000 Time:2018-08-21T11:58:14.2369850Z</Message> <HeaderName>x-ms-version</HeaderName> <HeaderValue>2021-04-10</HeaderValue> </Error>

Then, using node I installed Azurite version 3.16.0 which is the current version and the application would get through the startup process and then crash with a different error:

`RequestFailedException: Server failed to authenticate the request. Make sure the value of the Authorization header is formed correctly including the signature. RequestId:60676ef3-00d7-4db7-b762-dfb4496437cf Time:2022-04-12T17:28:28.637Z Status: 403 (Server failed to authenticate the request. Make sure the value of the Authorization header is formed correctly including the signature.) ErrorCode: AuthorizationFailure

Content:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?> <Error> AuthorizationFailure <Message>Server failed to authenticate the request. Make sure the value of the Authorization header is formed correctly including the signature. RequestId:60676ef3-00d7-4db7-b762-dfb4496437cf Time:2022-04-12T17:28:28.637Z</Message> </Error>

Headers: Server: Azurite-Blob/3.16.0 x-ms-error-code: AuthorizationFailure x-ms-request-id: 60676ef3-00d7-4db7-b762-dfb4496437cf Date: Tue, 12 Apr 2022 17:28:28 GMT Connection: keep-alive Keep-Alive: REDACTED Transfer-Encoding: chunked Content-Type: application/xml Azure.Storage.Blobs.BlobRestClient.DownloadAsync(string snapshot, string versionId, Nullable<int> timeout, string range, string leaseId, Nullable<bool> rangeGetContentMD5, Nullable<bool> rangeGetContentCRC64, string encryptionKey, string encryptionKeySha256, Nullable<EncryptionAlgorithmTypeInternal> encryptionAlgorithm, Nullable<DateTimeOffset> ifModifiedSince, Nullable<DateTimeOffset> ifUnmodifiedSince, string ifMatch, string ifNoneMatch, string ifTags, CancellationToken cancellationToken)

CryptographicException: An error occurred while trying to encrypt the provided data. Refer to the inner exception for more information. For more information go to http://aka.ms/dataprotectionwarning Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingBasedDataProtector.Protect(byte[] plaintext)`

Expected behavior

Expected behavior is for the application to start and use azurite as an emulator for local development as it does with version 12.10.0

Actual behavior

I have the stack trace for the second situation (using Azurite 3.16.0), the stack trace for the error when using 3.14.0 I no longer have access because installing Azurite via node removed the older version that was stored as an extension. There are 2 exceptions thrown.

Exception 1:

`Azure.RequestFailedException: Server failed to authenticate the request. Make sure the value of the Authorization header is formed correctly including the signature. RequestId:60676ef3-00d7-4db7-b762-dfb4496437cf Time:2022-04-12T17:28:28.637Z Status: 403 (Server failed to authenticate the request. Make sure the value of the Authorization header is formed correctly including the signature.) ErrorCode: AuthorizationFailure

Content:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?> <Error> AuthorizationFailure <Message>Server failed to authenticate the request. Make sure the value of the Authorization header is formed correctly including the signature. RequestId:60676ef3-00d7-4db7-b762-dfb4496437cf Time:2022-04-12T17:28:28.637Z</Message> </Error>

Headers: Server: Azurite-Blob/3.16.0 x-ms-error-code: AuthorizationFailure x-ms-request-id: 60676ef3-00d7-4db7-b762-dfb4496437cf Date: Tue, 12 Apr 2022 17:28:28 GMT Connection: keep-alive Keep-Alive: REDACTED Transfer-Encoding: chunked Content-Type: application/xml

at Azure.Storage.Blobs.BlobRestClient.DownloadAsync(String snapshot, String versionId, Nullable1 timeout, String range, String leaseId, Nullable1 rangeGetContentMD5, Nullable1 rangeGetContentCRC64, String encryptionKey, String encryptionKeySha256, Nullable1 encryptionAlgorithm, Nullable1 ifModifiedSince, Nullable1 ifUnmodifiedSince, String ifMatch, String ifNoneMatch, String ifTags, CancellationToken cancellationToken) at Azure.Storage.Blobs.Specialized.BlobBaseClient.StartDownloadAsync(HttpRange range, BlobRequestConditions conditions, Boolean rangeGetContentHash, Int64 startOffset, Boolean async, CancellationToken cancellationToken) at Azure.Storage.Blobs.Specialized.BlobBaseClient.DownloadStreamingInternal(HttpRange range, BlobRequestConditions conditions, Boolean rangeGetContentHash, IProgress1 progressHandler, String operationName, Boolean async, CancellationToken cancellationToken) at Azure.Storage.Blobs.Specialized.BlobBaseClient.DownloadStreamingAsync(HttpRange range, BlobRequestConditions conditions, Boolean rangeGetContentHash, IProgress1 progressHandler, CancellationToken cancellationToken) at Azure.Storage.Blobs.PartitionedDownloader.DownloadToAsync(Stream destination, BlobRequestConditions conditions, CancellationToken cancellationToken) at Azure.Storage.Blobs.Specialized.BlobBaseClient.StagedDownloadAsync(Stream destination, BlobRequestConditions conditions, IProgress1 progressHandler, StorageTransferOptions transferOptions, Boolean async, CancellationToken cancellationToken) at Azure.Storage.Blobs.Specialized.BlobBaseClient.DownloadToAsync(Stream destination, BlobRequestConditions conditions, StorageTransferOptions transferOptions, CancellationToken cancellationToken) at Azure.Extensions.AspNetCore.DataProtection.Blobs.AzureBlobXmlRepository.GetLatestDataAsync() at Azure.Extensions.AspNetCore.DataProtection.Blobs.AzureBlobXmlRepository.GetAllElementsAsync() at Azure.Extensions.AspNetCore.DataProtection.Blobs.AzureBlobXmlRepository.GetAllElements() at Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager.GetAllKeys() at Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingProvider.CreateCacheableKeyRingCore(DateTimeOffset now, IKey keyJustAdded) at Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingProvider.Microsoft.AspNetCore.DataProtection.KeyManagement.Internal.ICacheableKeyRingProvider.GetCacheableKeyRing(DateTimeOffset now) at Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingProvider.GetCurrentKeyRingCore(DateTime utcNow, Boolean forceRefresh) at Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingProvider.GetCurrentKeyRing() at Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingBasedDataProtector.Protect(Byte[] plaintext) CryptographicException: An error occurred while trying to encrypt the provided data. Refer to the inner exception for more information. For more information go to http://aka.ms/dataprotectionwarning Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingBasedDataProtector.Protect(byte[] plaintext) Microsoft.AspNetCore.Session.CookieProtection.Protect(IDataProtector protector, string data) Microsoft.AspNetCore.Session.SessionMiddleware.Invoke(HttpContext context) Serilog.AspNetCore.RequestLoggingMiddleware.Invoke(HttpContext httpContext) Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context) B3.Client.Web.SerilogFormDataMiddleware.Invoke(HttpContext context) in SerilogFormDataMiddleware.cs + await _next(context); B3.Client.Web.SerilogTenantIdMiddleware.Invoke(HttpContext context, Tenant tenant) in SerilogTenantIdMiddleware.cs + await _next(context); B3.Client.Web.SerilogUserIdMiddleware.Invoke(HttpContext context, UserManager<ApplicationUser> userManager) in SerilogUserIdMiddleware.cs + await _next(context); B3.Client.Web.Auth.TenantResolutionMiddleware.Invoke(HttpContext context, ITenantResolver tenantResolver, AdminContext adminContext) in TenantResolutionMiddleware.cs + await _next.Invoke(context); Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context) Microsoft.AspNetCore.Diagnostics.StatusCodePagesMiddleware.Invoke(HttpContext context) Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context)

Exception 2:

System.Security.Cryptography.CryptographicException: An error occurred while trying to encrypt the provided data. Refer to the inner exception for more information. For more information go to http://aka.ms/dataprotectionwarning —> Azure.RequestFailedException: Server failed to authenticate the request. Make sure the value of the Authorization header is formed correctly including the signature. RequestId:60676ef3-00d7-4db7-b762-dfb4496437cf Time:2022-04-12T17:28:28.637Z Status: 403 (Server failed to authenticate the request. Make sure the value of the Authorization header is formed correctly including the signature.) ErrorCode: AuthorizationFailure

Content:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?> <Error> AuthorizationFailure <Message>Server failed to authenticate the request. Make sure the value of the Authorization header is formed correctly including the signature. RequestId:60676ef3-00d7-4db7-b762-dfb4496437cf Time:2022-04-12T17:28:28.637Z</Message> </Error>

Headers: Server: Azurite-Blob/3.16.0 x-ms-error-code: AuthorizationFailure x-ms-request-id: 60676ef3-00d7-4db7-b762-dfb4496437cf Date: Tue, 12 Apr 2022 17:28:28 GMT Connection: keep-alive Keep-Alive: REDACTED Transfer-Encoding: chunked Content-Type: application/xml

at Azure.Storage.Blobs.BlobRestClient.DownloadAsync(String snapshot, String versionId, Nullable1 timeout, String range, String leaseId, Nullable1 rangeGetContentMD5, Nullable1 rangeGetContentCRC64, String encryptionKey, String encryptionKeySha256, Nullable1 encryptionAlgorithm, Nullable1 ifModifiedSince, Nullable1 ifUnmodifiedSince, String ifMatch, String ifNoneMatch, String ifTags, CancellationToken cancellationToken) at Azure.Storage.Blobs.Specialized.BlobBaseClient.StartDownloadAsync(HttpRange range, BlobRequestConditions conditions, Boolean rangeGetContentHash, Int64 startOffset, Boolean async, CancellationToken cancellationToken) at Azure.Storage.Blobs.Specialized.BlobBaseClient.DownloadStreamingInternal(HttpRange range, BlobRequestConditions conditions, Boolean rangeGetContentHash, IProgress1 progressHandler, String operationName, Boolean async, CancellationToken cancellationToken) at Azure.Storage.Blobs.Specialized.BlobBaseClient.DownloadStreamingAsync(HttpRange range, BlobRequestConditions conditions, Boolean rangeGetContentHash, IProgress1 progressHandler, CancellationToken cancellationToken) at Azure.Storage.Blobs.PartitionedDownloader.DownloadToAsync(Stream destination, BlobRequestConditions conditions, CancellationToken cancellationToken) at Azure.Storage.Blobs.Specialized.BlobBaseClient.StagedDownloadAsync(Stream destination, BlobRequestConditions conditions, IProgress1 progressHandler, StorageTransferOptions transferOptions, Boolean async, CancellationToken cancellationToken) at Azure.Storage.Blobs.Specialized.BlobBaseClient.DownloadToAsync(Stream destination, BlobRequestConditions conditions, StorageTransferOptions transferOptions, CancellationToken cancellationToken) at Azure.Extensions.AspNetCore.DataProtection.Blobs.AzureBlobXmlRepository.GetLatestDataAsync() at Azure.Extensions.AspNetCore.DataProtection.Blobs.AzureBlobXmlRepository.GetAllElementsAsync() at Azure.Extensions.AspNetCore.DataProtection.Blobs.AzureBlobXmlRepository.GetAllElements() at Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager.GetAllKeys() at Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingProvider.CreateCacheableKeyRingCore(DateTimeOffset now, IKey keyJustAdded) at Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingProvider.Microsoft.AspNetCore.DataProtection.KeyManagement.Internal.ICacheableKeyRingProvider.GetCacheableKeyRing(DateTimeOffset now) at Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingProvider.GetCurrentKeyRingCore(DateTime utcNow, Boolean forceRefresh) at Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingProvider.GetCurrentKeyRing() at Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingBasedDataProtector.Protect(Byte[] plaintext) --- End of inner exception stack trace --- at Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingBasedDataProtector.Protect(Byte[] plaintext) at Microsoft.AspNetCore.Session.CookieProtection.Protect(IDataProtector protector, String data) at Microsoft.AspNetCore.Session.SessionMiddleware.Invoke(HttpContext context) at Serilog.AspNetCore.RequestLoggingMiddleware.Invoke(HttpContext httpContext) at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context) at B3.Client.Web.SerilogFormDataMiddleware.Invoke(HttpContext context) in C:\DataDrive\B3\B3.Client.Web\System\SerilogFormDataMiddleware.cs:line 26 at B3.Client.Web.SerilogTenantIdMiddleware.Invoke(HttpContext context, Tenant tenant) in C:\DataDrive\B3\B3.Client.Web\System\SerilogTenantIdMiddleware.cs:line 26 at B3.Client.Web.SerilogUserIdMiddleware.Invoke(HttpContext context, UserManager1 userManager) in C:\DataDrive\B3\B3.Client.Web\System\SerilogUserIdMiddleware.cs:line 27 at B3.Client.Web.Auth.TenantResolutionMiddleware.Invoke(HttpContext context, ITenantResolver tenantResolver, AdminContext adminContext) in C:\DataDrive\B3\B3.Client.Web\Auth\TenantResolutionMiddleware.cs:line 43 at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context) at Microsoft.AspNetCore.Diagnostics.StatusCodePagesMiddleware.Invoke(HttpContext context) at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context)

Reproduction Steps

For the first issue (with Azurite installed as an extension via VS2022) do this:

• Create a new ASP MVC Core Application
• Add Azurite for VS following the instructions at this MS page: https://docs.microsoft.com/en-us/azure/storage/common/storage-use-azurite?tabs=visual-studio
• Use the Azure SDK to access a blob, in my case the call to await container.ExistsAsync()

For the second issue do the same as above but then, using node install Azurite 3.16.0. VS will now automatically use this version. Additionally add the following code to startup.cs:

var dataProtectionUri = fileStorage.GetDataProtectionUri().GetAwaiter().GetResult(); services.AddDataProtection() .PersistKeysToAzureBlobStorage(new Uri(dataProtectionUri)) .SetApplicationName("AppName");

The code that is called in fileStorage.GetDataProtectionUri()

` private async Task<BlobContainerClient> CreateContainerIfNotExist(string containerName) { var container = _serviceClient.GetBlobContainerClient(containerName); if (!await container.ExistsAsync()) { await container.CreateIfNotExistsAsync(); }

        return container;
    }

    public async Task<string> GetDataProtectionUri()
    {
        var container = await CreateContainerIfNotExist("key-container");
        var blob = container.GetBlobClient("keys-client-app.xml");

        var sasBuilder = new BlobSasBuilder()
        {
            BlobContainerName = container.Name,
            BlobName = blob.Name,
            Resource = "b",
            StartsOn = DateTimeOffset.UtcNow,
            ExpiresOn = DateTimeOffset.UtcNow.AddYears(1)
        };

        sasBuilder.SetPermissions(BlobContainerSasPermissions.Read | BlobContainerSasPermissions.Write);
        string sasToken = sasBuilder.ToSasQueryParameters(new StorageSharedKeyCredential(_accountName, _accountKey)).ToString();

        return $"{blob.Uri.AbsoluteUri}?{sasToken}";
    }`

The service client is initialized using:

_serviceClient = new BlobServiceClient(cs);

and the connection string I am using is provided by the same page linked above:

"DefaultEndpointsProtocol=https;AccountName=devstoreaccount1;AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;BlobEndpoint=https://127.0.0.1:10000/devstoreaccount1;QueueEndpoint=https://127.0.0.1:10001/devstoreaccount1;

Environment

Visual studio 17.1.3 Windows 10

.Net core environment

2.2.207 [C:\Program Files\dotnet\sdk] 3.0.103 [C:\Program Files\dotnet\sdk] 3.1.417 [C:\Program Files\dotnet\sdk] 5.0.402 [C:\Program Files\dotnet\sdk] 5.0.406 [C:\Program Files\dotnet\sdk] 6.0.201 [C:\Program Files\dotnet\sdk]

.NET runtimes installed: Microsoft.AspNetCore.All 2.2.8 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All] Microsoft.AspNetCore.App 2.2.8 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 3.0.3 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 3.1.20 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 3.1.21 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 3.1.23 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 5.0.11 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 5.0.12 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 5.0.13 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 5.0.15 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 6.0.3 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App] Microsoft.NETCore.App 2.2.8 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.NETCore.App 3.0.3 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.NETCore.App 3.1.20 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.NETCore.App 3.1.21 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.NETCore.App 3.1.23 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.NETCore.App 5.0.11 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.NETCore.App 5.0.12 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.NETCore.App 5.0.13 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.NETCore.App 5.0.15 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.NETCore.App 6.0.3 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.WindowsDesktop.App 3.0.3 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App] Microsoft.WindowsDesktop.App 3.1.20 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App] Microsoft.WindowsDesktop.App 3.1.21 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App] Microsoft.WindowsDesktop.App 3.1.23 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App] Microsoft.WindowsDesktop.App 5.0.11 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App] Microsoft.WindowsDesktop.App 5.0.15 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App] Microsoft.WindowsDesktop.App 6.0.3 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App] Microsoft.WindowsDesktop.App 6.0.4 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

[aolszowka]

FYI If you are using Azurite that ships with Visual Studio 2022 run Visual Studio Installer to upgrade to the latest, as of 17.2.3 this was working for me (I was on a slightly older version, I can’t recall which). The older versions of Azurite did not support the 2021-06-08 version of the API.

[pharring]

Updating to the latest Azurite (installed via “npm install -g azurite”) fixed it for me too.