[BUG] Accessing blob container fails with auth error, when using Azure Front Door CDN

Library name and version

Azure.Storage.Blobs 12.15.0

Describe the bug

When creating a BlobServiceClient based on an Azure Front Door endpoint URL, in ~50% of the cases, it produces an Authentication exception.

Expected behavior

In my solution, I want to create a SAS Uri for a blob item that is accessed via an Azure Front Door (AFD) endpoint. The blob container cannot be publicly accessible and therefore the SAS is needed. After creating a SAS for a blob item, you can replace the hostname of the blob URL E.g ‘https://<NAME>.blob.core.windows.net’ (from the complete SAS) with the hostname of the AFD endpoint. E.g: ‘https:// <NAME>.z01.azurefd.net’. With this new URL, you can access the blob item (even download it in the browser) through the AFD endpoint and use CDN.

In code, it seemed a clumsy solution first to create a SAS and then replace the hostname of the URL in the code. After some searching, I found a solution in creating a ‘BlobServiceClient’ with a different service URL, which points to the AFD endpoint. At first glance, this seems great. With this, I only have to change the initialization of the blob object(s) in my startup logic and keep my logic in the application the same. Unfortunately, this is not always working. In half of the cases, it works and in the other half, an exception is thrown in the code.

Why is this exception thrown and should it work as I intend? Or, are there better alternatives for what I want to achieve?

Actual behavior

The exception that is produced is:

Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
RequestId:abba20e0-f01e-004d-1969-52f96d000000
Time:2023-03-09T09:26:42.6689043Z
Status: 403 (Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.)
ErrorCode: AuthenticationFailed

Additional Information:
AuthenticationErrorDetail: The MAC signature found in the HTTP request 'C7MEyGoAuJVMoVG7whS58c46CW/2JUlPEiQoylCt4ok=' is not the same as any computed signature. Server used following string to sign: 'GET

bytes=0-8388607
x-ms-client-request-id:9798c48e-0480-4e8e-990e-2a5f615ec103
x-ms-date:Thu, 09 Mar 2023 09:26:42 GMT
x-ms-return-client-request-id:true
x-ms-version:2021-12-02
/<NAME>/machineimages
comp:list
restype:container'.

Content:
<?xml version="1.0" encoding="utf-8"?><Error><Code>AuthenticationFailed</Code><Message>Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
RequestId:abba20e0-f01e-004d-1969-52f96d000000
Time:2023-03-09T09:26:42.6689043Z</Message><AuthenticationErrorDetail>The MAC signature found in the HTTP request 'C7MEyGoAuJVMoVG7whS58c46CW/2JUlPEiQoylCt4ok=' is not the same as any computed signature. Server used following string to sign: 'GET

bytes=0-8388607
x-ms-client-request-id:9798c48e-0480-4e8e-990e-2a5f615ec103
x-ms-date:Thu, 09 Mar 2023 09:26:42 GMT
x-ms-return-client-request-id:true
x-ms-version:2021-12-02
/<NAME>/machineimages
comp:list
restype:container'.</AuthenticationErrorDetail></Error>

Headers:
Date: Thu, 09 Mar 2023 09:26:42 GMT
Connection: keep-alive
x-ms-request-id: abba20e0-f01e-004d-1969-52f96d000000
x-ms-error-code: AuthenticationFailed
Access-Control-Expose-Headers: REDACTED
Access-Control-Allow-Origin: *
x-azure-ref: REDACTED
X-Cache: REDACTED
Content-Type: application/xml
Content-Length: 800

Reproduction Steps

The following C# code reproduces (sometimes) the exception:

using System;
using System.Threading.Tasks;
using Azure.Storage;
using Azure.Storage.Blobs;
using Azure.Storage.Blobs.Models;

namespace ConsoleApp23
{
 internal class Program
 {
 static async Task Main(string[] args)
 {
 try
 {
 //Note1: Add a random image to the 'machineimages' container before running the code
			
 var accountName = "mystoragetst01";
 var accountKey = "*****************************************************************";

 //Note3: This is the original blob url that works without problems
 //string serviceUrl = "https://mystoragetst01.blob.core.windows.net";

 //Note4: This is the Azure Front Door url (Update the after running the BICEP script!) which throw regularly (~50%) an exception when using this.
 string serviceUrl = "https://mystoragetst01-?????????.z01.azurefd.net"; //TODO: Update url based on AFD endpoint after running the BICEP script!
 

 string containerName = "machineimages";

 StorageSharedKeyCredential credential = new StorageSharedKeyCredential(accountName, accountKey);
 BlobServiceClient blobServiceClient = new BlobServiceClient(new Uri(serviceUrl), credential);
 BlobContainerClient containerClient = blobServiceClient.GetBlobContainerClient(containerName);

 await foreach (BlobItem blobItem in containerClient.GetBlobsAsync()) //--> This line will produce the exception: 'Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.'
 {
 Console.WriteLine(blobItem.Name);
 }
 }
 catch (Exception exception)
 {
 Console.WriteLine(exception);
 }
 }
 }
}

Project file:

<Project Sdk="Microsoft.NET.Sdk">

 <PropertyGroup>
 <OutputType>Exe</OutputType>
 <TargetFramework>net7.0</TargetFramework>
 </PropertyGroup>

 <ItemGroup>
 <PackageReference Include="Azure.Storage.Blobs" Version="12.15.0" />
 </ItemGroup>

</Project>

The following BICEP script creates the Storage Account and the Azure Front Door (I composed it from multiple files so it isn’t production code):

param location string = resourceGroup().location

var storageAccountName = 'mystoragetst01'

resource storageAccountResource 'Microsoft.Storage/storageAccounts@2022-09-01' = {
 name: storageAccountName
 location: location
 kind: 'StorageV2'
 sku: {
 name: 'Standard_LRS'
 }
 properties: {
 minimumTlsVersion: 'TLS1_2'
 allowBlobPublicAccess: false
 allowSharedKeyAccess: true
 supportsHttpsTrafficOnly: true
 accessTier: 'Hot'
 }
}

//CORS rules
var corsProperty = {
 corsRules:[
 {
 allowedOrigins: [ 
 '*' 
 ]
 allowedHeaders: [
 '*'
 ]
 allowedMethods: [
 'GET'
 ]
 maxAgeInSeconds: 0
 exposedHeaders: [
 '*'
 ]
 }
 ]
}

//Set Data protection/Enable soft delete for blobs
resource blobServicesResource 'Microsoft.Storage/storageAccounts/blobServices@2022-09-01' = {
 parent: storageAccountResource
 name: 'default'
 properties: {
 containerDeleteRetentionPolicy: {
 enabled: true
 days: 14
 }
 deleteRetentionPolicy: {
 enabled: true
 days: 14
 }
 cors: corsProperty
 }
}

//Create container: 
resource containerResource 'Microsoft.Storage/storageAccounts/blobServices/containers@2022-09-01' = {
 name: '${storageAccountName}/default/machineimages'
 dependsOn: [
 
 ]
}

//Set Lifecycle management
resource storageAccountName_default 'Microsoft.Storage/storageAccounts/managementPolicies@2022-09-01' = {
 name: '${storageAccountName}/default'
 dependsOn: [
 storageAccountResource
 ]
 properties: {
 policy: {
 rules: [
 {
 enabled: true
 name: 'DeleteBlobInMessagesAfter186Days'
 type: 'Lifecycle'
 definition: {
 actions: {
 baseBlob: {
 delete: {
 daysAfterModificationGreaterThan: 186
 }
 }
 }
 filters: {
 blobTypes: [
 'blockBlob'
 ]
 prefixMatch: [
 'messages'
 ]
 }
 }
 }
 ]
 }
 }
}

//Create Azure Front Door
var frontDoorName = 'MyAFDTest01'

resource azureFrontDoorResource 'Microsoft.Cdn/profiles@2022-11-01-preview' = {
 name: frontDoorName
 location: 'Global'
 sku: {
 name: 'Premium_AzureFrontDoor'
 }
 properties: {
 originResponseTimeoutSeconds: 60
 extendedProperties: {
 }
 }
}


//Retrieves host name by substring 'https://mystoragetst01.blob.core.windows.net/' into 'mystoragetst01.blob.core.windows.net'
var storageAccountHostName = substring(storageAccountResource.properties.primaryEndpoints.blob, 8, length(storageAccountResource.properties.primaryEndpoints.blob) - 8 - 1)

//Add Endpoint to Azure Front Door
var endpointName = storageAccountName
var originGroupName = 'MyStorageTstOriginGroup'
var originName = 'MyStorageTstOrigin'
var originHostName = storageAccountHostName
var endpointRouteName = 'MyStorageTstRoute'
var webApplicationFirewallPolicyName = 'MyWafTest01'
var webApplicationFirewallPolicyMode = 'Detection'
var securityPolicyName = 'MyStorageTstSecurityPolicy'

//AFD Endpoint
resource endpointResource 'Microsoft.Cdn/profiles/afdendpoints@2022-11-01-preview' = {
 parent: azureFrontDoorResource
 name: toLower(endpointName)
 location: 'Global'
 properties: {
 enabledState: 'Enabled'
 }
}

//ADF Origin Group
resource originGroupResource 'Microsoft.Cdn/profiles/origingroups@2022-11-01-preview' = {
 parent: azureFrontDoorResource
 name: originGroupName
 properties: {
 loadBalancingSettings: {
 sampleSize: 4
 successfulSamplesRequired: 3
 additionalLatencyInMilliseconds: 50
 }
 healthProbeSettings: {
 probePath: '/'
 probeRequestType: 'HEAD'
 probeProtocol: 'Http'
 probeIntervalInSeconds: 100
 }
 sessionAffinityState: 'Disabled'
 }
}

//ADF Origin
resource originResource 'Microsoft.Cdn/profiles/origingroups/origins@2022-11-01-preview' = {
 parent: originGroupResource
 name: originName
 properties: {
 hostName: toLower(originHostName)
 httpPort: 80
 httpsPort: 443
 originHostHeader: toLower(originHostName)
 priority: 1
 weight: 1000
 enabledState: 'Enabled'
 enforceCertificateNameCheck: true
 }
}

//ADF Endpoint Route
resource routeResource 'Microsoft.Cdn/profiles/afdendpoints/routes@2022-11-01-preview' = {
 parent: endpointResource
 name: endpointRouteName
 properties: {
 cacheConfiguration: {
 compressionSettings: {
 isCompressionEnabled: true
 contentTypesToCompress: [
 'application/eot'
 'application/font'
 'application/font-sfnt'
 'application/javascript'
 'application/json'
 'application/opentype'
 'application/otf'
 'application/pkcs7-mime'
 'application/truetype'
 'application/ttf'
 'application/vnd.ms-fontobject'
 'application/xhtml+xml'
 'application/xml'
 'application/xml+rss'
 'application/x-font-opentype'
 'application/x-font-truetype'
 'application/x-font-ttf'
 'application/x-httpd-cgi'
 'application/x-javascript'
 'application/x-mpegurl'
 'application/x-opentype'
 'application/x-otf'
 'application/x-perl'
 'application/x-ttf'
 'font/eot'
 'font/ttf'
 'font/otf'
 'font/opentype'
 'image/svg+xml'
 'text/css'
 'text/csv'
 'text/html'
 'text/javascript'
 'text/js'
 'text/plain'
 'text/richtext'
 'text/tab-separated-values'
 'text/xml'
 'text/x-script'
 'text/x-component'
 'text/x-java-source'
 ]
 }
 queryStringCachingBehavior: 'IgnoreQueryString'
 }
 customDomains: []
 originGroup: {
 id: originGroupResource.id
 }
 ruleSets: []
 supportedProtocols: [
 'Https'
 ]
 patternsToMatch: [
 '/*'
 ]
 forwardingProtocol: 'HttpsOnly'
 linkToDefaultDomain: 'Enabled'
 httpsRedirect: 'Disabled'
 enabledState: 'Enabled'
 }
}

//Create Web Application Firewall
resource frontdoorWebApplicationFirewallPolicyResource 'Microsoft.Network/frontdoorwebapplicationfirewallpolicies@2022-05-01' = {
 name: webApplicationFirewallPolicyName
 location: 'Global'
 sku: {
 name: 'Premium_AzureFrontDoor'
 }
 properties: {
 policySettings: {
 enabledState: 'Enabled'
 mode: webApplicationFirewallPolicyMode
 customBlockResponseStatusCode: 403
 requestBodyCheck: 'Enabled'
 }
 customRules: {
 rules: []
 }
 managedRules: {
 managedRuleSets: [
 {
 ruleSetType: 'Microsoft_DefaultRuleSet'
 ruleSetVersion: '2.0'
 ruleSetAction: 'Block'
 ruleGroupOverrides: []
 exclusions: []
 }
 {
 ruleSetType: 'Microsoft_BotManagerRuleSet'
 ruleSetVersion: '1.0'
 ruleGroupOverrides: []
 exclusions: []
 }
 ]
 }
 }
}

//Web Application Firewall connection to the Endpoint
resource securityPolicyResource 'Microsoft.Cdn/profiles/securitypolicies@2022-11-01-preview' = {
 parent: azureFrontDoorResource
 name: securityPolicyName
 properties: {
 parameters: {
 wafPolicy: {
 id: frontdoorWebApplicationFirewallPolicyResource.id
 }
 associations: [
 {
 domains: [
 {
 id: endpointResource.id
 }
 ]
 patternsToMatch: [
 '/*'
 ]
 }
 ]
 type: 'WebApplicationFirewall'
 }
 }
}

Environment

NET runtime version: .Net 6.0 & .Net 7.0 IDE and version: Microsoft Visual Studio Enterprise 2022 (64-bit) - Current Version 17.5.1 OS: Windows 11 Pro Version 22H2 (OS Build 22621.1344)

[navba-MSFT]

@StannieV Apologies for the late reply. Thanks for reaching out to us and reporting this issue. We are looking into this issue and we will provide an update.

[github-actions[bot]]

Hi @StannieV, we’re sending this friendly reminder because we haven’t heard back from you in 7 days. We need more information about this issue to help address it. Please be sure to give us your input. If we don’t hear back from you within 14 days of this comment the issue will be automatically closed. Thank you!