Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Unobserved task exception on gateway failure

See original GitHub issue

SDK Version: 3.35.2 The failure in backend (error gateway response) causes an unobserved task exception in the background gateway cache refreshing. Doesn’t fail the application work, but lands in an observer and may trigger alerts.

Maybe related to #3994

[SubscribeAndLogUnhandledExceptions,LogSourceExtensions.cs(27) <NoCorrelationId>] Unobserved task exception: System.Threading.Tasks.UnwrapPromise`1[System.Threading.Tasks.VoidTaskResult] {DocumentClientException("The requested resource is no longer available
at the server. ActivityId: 1e8489e3-2cda-41fb-afce-50852a4a31b9, Request URI: /9c967a2b-7b0d-4f49-a38d-36f1391fda86/49ad7700-95bd-4037-ac52-b32664b76756, RequestStats: , SDK: Microsoft.Azure.Documents.Common/2.14.0, Microsoft.Azure.Documents.Common/2.14.0, Windows/10.0.20348 cosmos-netstandard-sdk/3.30.8") Source: 'Microsoft.Azure.Cosmos.Client'
at Microsoft.Azure.Cosmos.GatewayStoreClient.ParseResponseAsync(HttpResponseMessage responseMessage, JsonSerializerSettings serializerSettings, DocumentServiceRequest request)
at Microsoft.Azure.Cosmos.Routing.GatewayAddressCache.GetServerAddressesViaGatewayAsync(DocumentServiceRequest request, String collectionRid, IEnumerable`1 partitionKeyRangeIds, Boolean forceRefresh)
at Microsoft.Azure.Cosmos.Routing.GatewayAddressCache.GetAddressesForRangeIdAsync(DocumentServiceRequest request, PartitionAddressInformation cachedAddresses, String collectionRid, String partitionKeyRangeId, Boolean forceRefresh)
at Microsoft.Azure.Cosmos.AsyncCacheNonBlocking`2.AsyncLazyWithRefreshTask`1.CreateAndWaitForBackgroundRefreshTaskAsync(Func`2 createRefreshTask)
at Microsoft.Azure.Cosmos.AsyncCacheNonBlocking`2.UpdateCacheAndGetValueFromBackgroundTaskAsync(TKey key, AsyncLazyWithRefreshTask`1 initialValue, Func`2 callbackDelegate, String operationName)
at Microsoft.Azure.Cosmos.AsyncCacheNonBlocking`2.RefreshAsync(TKey key, Func`2 singleValueInitFunc)
at Microsoft.Azure.Cosmos.Routing.GatewayAddressCache.<>c__DisplayClass25_0.<<TryGetAddressesAsync>b__5>d.MoveNext() }
=== Error logged at ===
at Microsoft.Skype.Pstn.Common.Logging.LogSource.Critical(Exception exception, InterpolatedString& message) in /_/Microsoft.Skype.Pstn.Common/Logging/LogSource.cs:line 526
at Microsoft.Skype.Pstn.Common.Logging.LogSourceExtensions.<>c__DisplayClass0_0.<SubscribeAndLogUnhandledExceptions>b__1(Object sender, UnobservedTaskExceptionEventArgs args) in /_/Microsoft.Skype.Pstn.Common/Logging/LogSourceExtensions.cs:line 27
at System.Threading.Tasks.TaskExceptionHolder.Finalize()

Issue Analytics

State:
Created 2 months ago
Reactions:1
Comments:5 (2 by maintainers)

Top GitHub Comments

1reaction

cizracommented, Aug 11, 2023

No, there’s no spike of misbehavior. It’s just that we finally took the time to dig in and see who is generating those failures.

We have a monitor that monitors UnobservedTaskExceptions (with an urgent alert attached), so we keep getting alerts from this bug, every now and then.

0reactions

cizracommented, Aug 16, 2023

@ealsur I think this should do the trick:

            TaskScheduler.UnobservedTaskException += (sender, args) =>
            {
                log.Critical(args.Exception.Unwrap(), $"Unobserved task exception: {sender?.GetType()}");
            };