TaskExtensions.WithCancellationAndTimeout leave a trail of unobserved Task exceptions when the cancelled or timed out.

The issue

As has been mentioned in https://github.com/dotnet/runtime/issues/61180:

an application records all TaskScheduler.UnobservedTaskException events to make sure that there are no hidden bugs and all asynchronous components’ life cycles are controlled. The failing thread’s callstack is rooted in the framework’s own code, although it could be difficult to say if it has anything to do with the application or not because the operation is Task-based.

System.AggregateException: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (Name or service not known)
---> System.Net.Internals.SocketExceptionFactory+ExtendedSocketException (00000005, 0xFFFDFFFF): Name or service not known
at System.Net.Dns.GetHostEntryOrAddressesCore(String hostName, Boolean justAddresses)
at System.Net.Dns.<>c.<GetHostEntryOrAddressesCoreAsync>b__27_2(Object s)
at System.Threading.Tasks.Task`1.InnerInvoke()
at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location ---at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot, Thread threadPoolThread)
--- End of inner exception stack trace ---

I have been quick to suspect a bug in GetHostEntryOrAddressesCoreAsync of .Net framework but that was a mistake. Have been advised to look out for “helper” methods that preempt a Task that GetHostEntryOrAddressesCoreAsync creates.

I’ve found such a helper method while taking a look at the npgsql library code.

TaskExtensions.cs helper class

When Dns.GetHostAddressesAsync(Host) creates a Task in https://github.com/npgsql/npgsql/blob/b3a534517bfe5fc1441b714f4db799c7a4fe04db/src/Npgsql/Internal/NpgsqlConnector.cs#L1014 and passes it to TaslkExtensions.WithCancellation and the CancellationToken is cancelled before the DNS Task completion then the Task is left to its own devices and triggers unobserved Task exception notification if fails later.

Steps to reproduce

I desided to validate the helper method’s general behavior by copying it into a test that wraps a Task with a delayed failure to get 100% consistent result. Trying the same with GetHostEntryOrAddressesCoreAsync could be tricky.

The test results in

System.AggregateException
A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (TestException)
  Exception doesn't have a stacktrace

System.Exception
TestException

The test:

        /// <summary>
        /// A copy of TaslkExtensions.WithCancellation method that is being tested
        /// taken from https://github.com/npgsql/npgsql/blob/059c2499f1f92e7643d65ae32afb114e584dbee3/src/Npgsql/TaskExtensions.cs
        /// </summary>
        static async Task WithCancellation(Task task, CancellationToken cancellationToken)
        {
            var tcs = new TaskCompletionSource<bool>();
            using (cancellationToken.Register(s => ((TaskCompletionSource<bool>)s!).TrySetResult(true), tcs))
            {
                if (task != await Task.WhenAny(task, tcs.Task))
                {
                    throw new TaskCanceledException(task);
                }
            }
            await task;
        }

        /// <summary>
        /// The test creates a delayed execution Task that fails subsequently and triggers 'TaskScheduler.UnobservedTaskException event'.
        /// </summary>
        [Fact]
        public async Task TaskExtensions_WithCancellation_TestAsync()
        {
            Exception unobservedTaskException = null;

            // Subscribe to UnobservedTaskException event to store the Exception, if any.
            TaskScheduler.UnobservedTaskException += (_, args) =>
            {
                if (!args.Observed)
                {
                    args.SetObserved();
                }
                unobservedTaskException = args.Exception;
            };

            // Invoke the method that creates a delayed execution Task preemptableTask that fails subsequently.
            await CreateTaskAndPreemptWithCancellationAsync();

            // Wait enough time for the preemptableTask task to fail and then do the GC collect to trigger the finalizer.
            await Task.Delay(2000);
            GC.Collect();
            GC.WaitForPendingFinalizers();

            // Verify the unobserved Task exception event has been received.
            if (unobservedTaskException is not null)
            {
                throw new Exception(unobservedTaskException.Message, unobservedTaskException);
            }
        }

        /// <summary>
        /// Create a delayed execution Task preemptableTask that fails subsequently (after preemptableTask goes out of scope).
        /// </summary>
        static async Task CreateTaskAndPreemptWithCancellationAsync()
        {
            Task preemptableTask = Task.Delay(500, CancellationToken.None).ContinueWith(_ => throw new Exception("TestException"),TaskContinuationOptions.ExecuteSynchronously);
            using var cts = new CancellationTokenSource(100);
            try
            {
                await WithCancellation(preemptableTask, cts.Token);
            }
            catch (OperationCanceledException)
            {
                Assert.True(cts.IsCancellationRequested);
            }
            Assert.False(preemptableTask.IsCompleted);
        }

Further technical details

Npgsql version: 5.0 or below PostgreSQL version: Any Operating system: Any

Acceptance criteria

• An update to npgsql testing framework that ensures that the library does not leave a trail of unobserved Tasks with unexpected failures.
• Update the helper method(s) to allow for both long running Tasks preemption and at the same time not to leave the preempted Task unobserved. One strategy could be placing the abandoned tasks into a container that waits until the tasks reach a state of completion, then check the result: whether it can be ignored (a transient error) or not (a programmer’s bug).

[roji]

Thanks @baal2000!

[baal2000]

Agreed. As this is not urgent I’ll take my time as this is first time attempting to make a change in a public repo.