Lightrun
question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

[BUG] BlobChangeFeedClient.GetChangesAsync returns enumerator that throws System.FormatException

See original GitHub issue

Describe the bug I am attempting to enumerate through the Change Feed of an Azure Storage Container.

Initially it was working. When it was working, the only changes I had made to the storage account was to upload two blobs via the Azure Portal.

I then uploaded a file to a block blob using Powershell. Now the c# code to read the change feed throws an exception.

Expected behavior I should be able to read through the change feed regardless of which tool I used to make the changes.

Actual behavior (include Exception or Stack Trace) An exception gets thrown. Here is the stack trace:

   at System.Guid.GuidResult.SetFailure(Boolean overflow, String failureMessageID)
   at System.Guid.TryParseExactD(ReadOnlySpan`1 guidString, GuidResult& result)
   at System.Guid.TryParseGuid(ReadOnlySpan`1 guidString, GuidResult& result)
   at System.Guid.Parse(String input)
   at Azure.Storage.Blobs.ChangeFeed.BlobChangeFeedEventData..ctor(Dictionary`2 record)
   at Azure.Storage.Blobs.ChangeFeed.BlobChangeFeedEvent..ctor(Dictionary`2 record)
   at Azure.Storage.Blobs.ChangeFeed.Chunk.<Next>d__15.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
   at Azure.Storage.Blobs.ChangeFeed.Shard.<Next>d__10.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at Azure.Storage.Blobs.ChangeFeed.Segment.<GetPage>d__13.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at Azure.Storage.Blobs.ChangeFeed.ChangeFeed.<GetPage>d__14.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at Azure.Storage.Blobs.ChangeFeed.BlobChangeFeedAsyncPageable.<AsPages>d__6.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Threading.Tasks.Sources.ManualResetValueTaskSourceCore`1.GetResult(Int16 token)
   at Azure.Storage.Blobs.ChangeFeed.BlobChangeFeedAsyncPageable.<AsPages>d__6.System.Threading.Tasks.Sources.IValueTaskSource<System.Boolean>.GetResult(Int16 token)
   at Azure.AsyncPageable`1.<GetAsyncEnumerator>d__6.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Azure.AsyncPageable`1.<GetAsyncEnumerator>d__6.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Threading.Tasks.Sources.ManualResetValueTaskSourceCore`1.GetResult(Int16 token)
   at Azure.AsyncPageable`1.<GetAsyncEnumerator>d__6.System.Threading.Tasks.Sources.IValueTaskSource<System.Boolean>.GetResult(Int16 token)
   at System.Runtime.CompilerServices.ValueTaskAwaiter`1.GetResult()
   at Webpack2.Controllers.ChangeFeedController.<ChangeFeedAsync>d__1.MoveNext() in C:\Users\sheph\Documents\Development\asp.net-core\SignalRWalkthrough\Webpack2\Controllers\ChangeFeedController.cs:line 31

To Reproduce Steps to reproduce the behavior (include a code snippet, screenshot, or any additional information that might help us reproduce the issue)

  1. Enable the change feed on your storage account.
  2. Upload a file using Azure Powershell. Here is the code via which I did this:
Set-AzStorageBlobContent -File ".\az-100.jpg" -Container $container.Name -Blob "az-100.jpg" -BlobType Block -Metadata @{"foo"= "bar"} -Context $ctx
  1. Run c# code like the following:
        public async Task<List<BlobChangeFeedEvent>> ChangeFeedAsync()
        {
            // Get a new blob service client.
            var blobServiceClient = new BlobServiceClient(ConnectionString);

            // Get a new change feed client.
            BlobChangeFeedClient changeFeedClient = blobServiceClient.GetChangeFeedClient();

            List<BlobChangeFeedEvent> changeFeedEvents = new List<BlobChangeFeedEvent>();

            // Get all the events in the change feed.
            Azure.AsyncPageable<BlobChangeFeedEvent> asyncChangeFeedEvents = changeFeedClient.GetChangesAsync();
            var enumerator = asyncChangeFeedEvents.GetAsyncEnumerator();
            try
	    {
                bool continueReading = await enumerator.MoveNextAsync();  // <<=== Exception gets thrown here
                while (continueReading)
                {
                    var currentItem = enumerator.Current;
                    changeFeedEvents.Add(currentItem);
                    continueReading = await enumerator.MoveNextAsync();
                }
            }
            catch(Exception ex)
	    {
                System.Diagnostics.Debug.WriteLine(ex.ToString());
	    }
            return changeFeedEvents;
        }

Environment:

  • I am using the following packages
  <ItemGroup>
    <PackageReference Include="Azure.Storage.Blobs" Version="12.7.0" />
    <PackageReference Include="Azure.Storage.Blobs.ChangeFeed" Version="12.0.0-preview.7" />
  </ItemGroup>
  • I am running a .NET 5.0 application. Invoking dotnet --info returns the following:
.NET SDK (reflecting any global.json):
 Version:   5.0.101
 Commit:    d05174dc5a

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.19041
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\5.0.101\

Host (useful for support):
  Version: 5.0.1
  Commit:  b02e13abab

.NET SDKs installed:
  2.1.4 [C:\Program Files\dotnet\sdk]
  2.1.202 [C:\Program Files\dotnet\sdk]
  2.1.508 [C:\Program Files\dotnet\sdk]
  2.1.700 [C:\Program Files\dotnet\sdk]
  2.1.801 [C:\Program Files\dotnet\sdk]
  2.1.802 [C:\Program Files\dotnet\sdk]
  2.2.300 [C:\Program Files\dotnet\sdk]
  2.2.401 [C:\Program Files\dotnet\sdk]
  2.2.402 [C:\Program Files\dotnet\sdk]
  3.0.100 [C:\Program Files\dotnet\sdk]
  5.0.101 [C:\Program Files\dotnet\sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.All 2.1.11 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.1.12 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.1.13 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.1.23 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.5 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.7 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.8 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.1.11 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.1.12 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.1.13 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.1.23 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.5 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.7 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.8 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.0.0 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.10 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 5.0.1 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 2.0.5 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.0.9 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.11 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.12 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.13 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.23 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.5 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.7 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.8 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.0.0 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.10 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 5.0.1 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.WindowsDesktop.App 3.0.0 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 3.1.10 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 5.0.1 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

To install additional .NET runtimes or SDKs:
  https://aka.ms/dotnet-download
  • My IDE is Visual Studio 2019 Version 16.8.3

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:6 (3 by maintainers)

github_iconTop GitHub Comments

1reaction
AndrewShepherdcommented, Dec 26, 2020

I downloaded the .avro file, and viewed it through this nifty avro-viewer online service: https://zymeworks.github.io/avro-viewer/

I can see that Powershell generated a client request ID of Azure-Storage-PowerShell-3a5581dd-8f8c-4f0b-9cbc-afae8e73459e

I can also see that the c# library code is attempting to parse this as a GUID.

https://github.com/Azure/azure-sdk-for-net/blob/316b3baed5219ff2be8f2e47f382c34a26e639f4/sdk/storage/Azure.Storage.Blobs.ChangeFeed/src/Models/BlobChangeFeedEventData.cs#L23

The code cannot operate under the assumption that the client request ID’s will be valid guid values, when the storage API’s do not enforce this.

0reactions
seanmcc-msftcommented, Jan 5, 2021

@tg-msft, ChangeFeed is still in preview, can we just change ClientRequestId to a string?

Read more comments on GitHub >

github_iconTop Results From Across the Web

[BUG] foreach Azure.Pageable<BlobItem> then throw ...
It throws the "Unhandled exception. System.FormatException: String '' was not recognized as a valid DateTime." Expected behavior. I think it can ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found
Previous page

[BUG] Blob container's metadata is always empty when listed

Next page

[QUERY] How to query for keys on demand during runtime?