[BUG] Getting 404 responses from Azure BlobClient during a manual failover for RA-GRS Storage Account

Library name and version

Azure.Storage.Blob 12.8.0

Describe the bug

We do have a Storage Account with RA-GRS replication type. Also, we have set GeoRedundantSecondaryUri to be a secondary Blob Service Uri. Then we try to perform a manual failover for the Storage Account. We are getting 404 responses without retries and any attempts to read from the secondary location.

Expected behavior

Library tries to read from the secondary blob Uri

Actual behavior

The response code is 404. Blob Client doesn’t try any retries from the secondary blob service endpoint

Reproduction Steps

1. Storage Account with RA-GRS replication type
2. Set GeoRedundantSecondaryUri to be a secondary blob service endpoint
3. Upload a file to a blob storage
4. Try to read previously uploaded file continuously
5. Perform a manual failover for the storage account
6. Read request will start to fail with 404 (NotFound) Status Code

Environment

.NET SDK (reflecting any global.json):
 Version:   6.0.201
 Commit:    ef40e6aa06

Runtime Environment:
 OS Name:     Mac OS X
 OS Version:  10.15
 OS Platform: Darwin
 RID:         osx.10.15-x64
 Base Path:   /usr/local/share/dotnet/sdk/6.0.201/

Host (useful for support):
  Version: 6.0.3
  Commit:  c24d9a9c91

[amnguye]

Hi, Looks like this error is not caused by the SDK. A 404 BlobNotFound Error could mean anything from authentication issue or to the failover replication not being successful.

I would recommend opening up an Azure Support Request to check in the blob in question that’s giving the 404 error. The SDK is only passing on the error to the client, not necessarily causing the 404 to happen in the first place. https://docs.microsoft.com/en-us/azure/azure-portal/supportability/how-to-create-azure-support-request

[RakeshMohanMSFT]

@ykulyk404 Thank you for reaching out, we are looking into it.