Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Deadlock in low thread environment

See original GitHub issue

I use linq2db.EntityFramewortkCore (6.7.1) in my read layer in ASP MVC application. I limit amount of threads in ThreadPool with ThreadPool.SetMaxThreads(12, 100). Recently I witnessed that my application stops responding to any HTTP request. By taking dump I found that most threads blocks in linq2db (deadlock):

[HelperMethodFrame_1OBJ: 00007fbed4974db0] System.Threading.Monitor.ObjWait(Int32, System.Object)
System.Threading.ManualResetEventSlim.Wait(Int32, System.Threading.CancellationToken) [/_/src/libraries/System.Private.CoreLib/src/System/Threading/ManualResetEventSlim.cs @ 570]
System.Threading.Tasks.Task.SpinThenBlockingWait(Int32, System.Threading.CancellationToken) [/_/src/libraries/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs @ 2985]
System.Threading.Tasks.Task.InternalWaitCore(Int32, System.Threading.CancellationToken) [/_/src/libraries/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs @ 2920]
System.Threading.Tasks.Task`1[[System.__Canon, System.Private.CoreLib]].GetResultCore(Boolean) [/_/src/libraries/System.Private.CoreLib/src/System/Threading/Tasks/Future.cs @ 467]
*** LinqToDB.EntityFrameworkCore.Internal.LinqToDBForEFQueryProvider`1[[System.__Canon, System.Private.CoreLib]].GetAsyncEnumerator(System.Threading.CancellationToken)
System.Runtime.CompilerServices.ConfiguredCancelableAsyncEnumerable`1[[System.__Canon, System.Private.CoreLib]].GetAsyncEnumerator() [/_/src/libraries/System.Private.CoreLib/src/System/Runtime/CompilerServices/ConfiguredCancelableAsyncEnumerable.cs @ 42]
Microsoft.EntityFrameworkCore.EntityFrameworkQueryableExtensions+<ToListAsync>d__65`1[[System.__Canon, System.Private.CoreLib]].MoveNext()
System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[Microsoft.EntityFrameworkCore.EntityFrameworkQueryableExtensions+<ToListAsync>d__65`1[[System.__Canon, System.Private.CoreLib]], Microsoft.EntityFrameworkCore]](<ToListAsync>d__65`1<System.__Canon> ByRef) [/_/src/libraries/System.Private.CoreLib/src/System/Runtime/CompilerServices/AsyncMethodBuilderCore.cs @ 38]
Microsoft.EntityFrameworkCore.EntityFrameworkQueryableExtensions.ToListAsync[[System.__Canon, System.Private.CoreLib]](System.Linq.IQueryable`1<System.__Canon>, System.Threading.CancellationToken)
LinqToDB.EntityFrameworkCore.LinqToDBExtensionsAdapter.ToListAsync[[System.__Canon, System.Private.CoreLib]](System.Linq.IQueryable`1<System.__Canon>, System.Threading.CancellationToken)
LinqToDB.AsyncExtensions+<ToListAsync>d__8`1[[System.__Canon, System.Private.CoreLib]].MoveNext()
System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[LinqToDB.AsyncExtensions+<ToListAsync>d__8`1[[System.__Canon, System.Private.CoreLib]], linq2db]](<ToListAsync>d__8`1<System.__Canon> ByRef) [/_/src/libraries/System.Private.CoreLib/src/System/Runtime/CompilerServices/AsyncMethodBuilderCore.cs @ 38]
LinqToDB.AsyncExtensions.ToListAsync[[System.__Canon, System.Private.CoreLib]](System.Linq.IQueryable`1<System.__Canon>, System.Threading.CancellationToken)
App.Database.Repositories.MoviesRepository+<GetMovies>d__2.MoveNext()
-- earlier frames omitted

This callstack originates from simple code similar to this:

var localResult = await sortQuery
    .AsNoTracking()
    .ToLinqToDB()
    .ToListAsyncLinqToDB();

Code investigation brings clear sync over async pattern that causes this in LinqToDBForEFQueryProvider

public IAsyncEnumerator<T> GetAsyncEnumerator(CancellationToken cancellationToken)
{
    return Task.Run(() => QueryProvider.ExecuteAsyncEnumerable<T>(Expression, cancellationToken),
    cancellationToken).Result.GetAsyncEnumerator(cancellationToken);
}

There was attempt to fix this issue in https://github.com/linq2db/linq2db.EntityFrameworkCore/commit/ea55615a655ae0b91e90bb8db7da15fcc4b4d546 but new fix does not help too much in my case.

I patched LinqToDBExtensionsAdapter.ToListAsync with this code:

public async Task<List<TSource>> ToListAsync<TSource>(
	IQueryable<TSource> source,
	CancellationToken token)
{
	var list = new List<TSource>();
	var provider = source.Provider as LinqToDBForEFQueryProvider<TSource>;
	var enumerator = await provider!.ExecuteAsyncEnumerable<TSource>(provider.Expression, token).ConfigureAwait(false);
	
	await foreach (var element in enumerator.WithCancellation(token))
	{
		list.Add(element);
	}

	return list;
}

and it immediately helps but I think this is a dirty hack.

Any idea how this can be fixed in another way?