question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

[BUG] SyncCopyFromUriAsync() requires a SAS uri while StartCopyFromUriAsync() does not

See original GitHub issue

Library name and version

Azure.Storage.Blobs 12.14.1

Describe the bug

We are trying to use SyncCopyFromUriAsync() as a replacement of StartCopyFromUriAsync() / WaitForCompletionAsync() pair. (as a note: we do know about 256mb limit and our files are known to be less then 1 mb).

We have a working code like

	// Copy op: ok
	var sourceClient = containerClient.GetBlobClient("source.txt");
	var destinationClient = containerClient.GetBlobClient("destination.txt");
	//... 
	var copyOp = await destinationClient.StartCopyFromUriAsync(sourceClient.Uri);
	await copyOp.WaitForCompletionAsync();

and it breaks if we do use SyncCopyFromUriAsync() instead:

	// Sync copy: throws CannotVerifyCopySource error
	var sourceClient = containerClient.GetBlobClient("source.txt");
	var destinationClient = containerClient.GetBlobClient("destination.txt");
	//... 
	await destinationClient.SyncCopyFromUriAsync(sourceClient.Uri);

	// Sync copy with SAS: ok
	//await destinationClient.SyncCopyFromUriAsync(sourceClient.GenerateSasUri(BlobSasPermissions.Read, DateTimeOffset.UtcNow.AddMinutes(5)));

Expected behavior

We do expect the SyncCopyFromUriAsync() should mirror the StartCopyFromUriAsync() contract and should accept non-SAS uris for blobs from the same container.

Actual behavior

The SyncCopyFromUriAsync() throws RequestFailedException:

Azure.RequestFailedException : Server failed to authenticate the request. Please refer to the information in the www-authenticate header.
RequestId:5b205f18-901e-0049-5837-e9557d000000
Time:2022-10-26T12:36:25.0532752Z
Status: 401 (Server failed to authenticate the request. Please refer to the information in the www-authenticate header.)
ErrorCode: CannotVerifyCopySource

while trying to copy blob from the same container

Reproduction Steps

	public sealed class BlobStorageTroubleshootingTests1
	{
		private const string StorageConnectionString = "DefaultEndpointsProtocol=https;AccountName=...;AccountKey=...;EndpointSuffix=core.windows.net";

		[Test]
		public async Task TestSyncCopy_Ok()
		{
			var blobServiceClient = new BlobServiceClient(StorageConnectionString);
			var containerClient = blobServiceClient.GetBlobContainerClient("delete-me-" + Guid.NewGuid().ToString("N"));
			var sourceClient = containerClient.GetBlobClient("source.txt");
			var destinationClient = containerClient.GetBlobClient("destination.txt");

			var exists = await containerClient.ExistsAsync();
			if (!exists)
				await containerClient.CreateAsync();

			try
			{
				using var content = new MemoryStream(Encoding.UTF8.GetBytes("Test content"));

				await sourceClient.UploadAsync(content);

				// Sync copy: throws CannotVerifyCopySource error
				await destinationClient.SyncCopyFromUriAsync(sourceClient.Uri);

				// Sync copy with SAS: ok
				//await destinationClient.SyncCopyFromUriAsync(sourceClient.GenerateSasUri(BlobSasPermissions.Read, DateTimeOffset.UtcNow.AddMinutes(5)));

				using var download = (await destinationClient.DownloadStreamingAsync()).Value;
				var expectedText = await new StreamReader(download.Content).ReadToEndAsync();

				Assert.AreEqual(expectedText, "Test content");
			}
			finally
			{
				if (!exists)
					await containerClient.DeleteAsync();
			}
		}

		[Test]
		public async Task TestCopyOp_Ok()
		{
			var blobServiceClient = new BlobServiceClient(StorageConnectionString);
			var containerClient = blobServiceClient.GetBlobContainerClient("delete-me-" + Guid.NewGuid().ToString("N"));
			var sourceClient = containerClient.GetBlobClient("source.txt");
			var destinationClient = containerClient.GetBlobClient("destination.txt");

			var exists = await containerClient.ExistsAsync();
			if (!exists)
				await containerClient.CreateAsync();

			try
			{
				using var content = new MemoryStream(Encoding.UTF8.GetBytes("Test content"));

				await sourceClient.UploadAsync(content);

				// Copy op: ok
				var copyOp = await destinationClient.StartCopyFromUriAsync(sourceClient.Uri);
				await copyOp.WaitForCompletionAsync();

				using var download = (await destinationClient.DownloadStreamingAsync()).Value;
				var expectedText = await new StreamReader(download.Content).ReadToEndAsync();

				Assert.AreEqual(expectedText, "Test content");
			}
			finally
			{
				if (!exists)
					await containerClient.DeleteAsync();
			}
		}
	}

Environment

 OS Name:     Windows
 OS Version:  10.0.22000
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\6.0.402\

Host:
  Version:      6.0.10
  Architecture: x64
  Commit:       5a400c212a

.NET SDKs installed:
  3.1.424 [C:\Program Files\dotnet\sdk]
  6.0.203 [C:\Program Files\dotnet\sdk]
  6.0.305 [C:\Program Files\dotnet\sdk]
  6.0.402 [C:\Program Files\dotnet\sdk]

Issue Analytics

  • State:closed
  • Created a year ago
  • Comments:5 (3 by maintainers)

github_iconTop GitHub Comments

1reaction
navba-MSFTcommented, Oct 28, 2022

@ig-sinicyn Thanks for getting back. We will now proceed with closure of this GitHub issue. If you need any further assistance on this issue in future, please feel free to reopen this thread. We would be happy to help.

1reaction
navba-MSFTcommented, Oct 28, 2022

@ig-sinicyn I looked at the backend logs for the below error that you had encountered with SyncCopyFromUriAsync():

Azure.RequestFailedException : Server failed to authenticate the request. Please refer to the information in the www-authenticate header.
RequestId:5b205f18-901e-0049-5837-e9557d000000
Time:2022-10-26T12:36:25.0532752Z
Status: 401 (Server failed to authenticate the request. Please refer to the information in the www-authenticate header.)
ErrorCode: CannotVerifyCopySource

The SyncCopyFromUriAsync function uses Copy Blob Rest API underneath. So this behavior seems to be by-design and the same has been documented.

The source for a Copy Blob From URL operation can be any committed block blob in any Azure storage account that’s either public or authorized with a shared access signature.

More Info:

x-ms-copy-source:name Required. Specifies the URL of the source blob. The value can be a URL of up to 2 kibibytes (KiB) in length that specifies a blob. The value should be URL-encoded as it would appear in a request URI. The source blob must either be public or be authorized via a shared access signature. If the source blob is public, no authorization is required to perform the operation.

So if you don’t wish to provide the SAS token, try to make the Access Level on the container to public. I tried to make the access level of container to public (from private) SyncCopyFromUriAsync( ) worked without the SAS token.

Read more comments on GitHub >

github_iconTop Results From Across the Web

BlockBlobClient Class (Azure.Storage.Blobs.Specialized)
The BlockBlobClient allows you to manipulate Azure Storage block blobs. Block blobs let you upload large blobs efficiently. Block blobs are comprised of ......
Read more >
cannot access to URI SAS blob storage AuthenticationFailed
hi, im trying to access to a specific blob by URI SAS but i got error AuthenticationErrorDetail, is there anything im doing wrong...
Read more >
Unable to get Azure Blob Client StartCopyFromUriAsync to ...
I either get an Exception or get an error stating that "The operation has not completed yet.". The BlobCopyStatus is pending until the ......
Read more >
Copying Blobs with the V12 Storage SDK
The reason the method is called StartCopyFromUriAsync , is that it doesn't necessarily finish instantaneously, and that will be the case if you ......
Read more >
Startcopyfromuriasync
This sample code uses the StartCopyFromUriAsync () method to perform the copy ... But it doesn't have the information it needs to do...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found