Could not find free slot in Npgsql.NpgsqlConnector[] when closing.

Steps to reproduce

Unfortunately, none really. It is such a random error that I currently can’t narrow it down to a specific issue.

The issue

We are getting the exception below all over the place in different applications. It happens multiple times a day, but given the number of requests it is actually a pretty rare occasion. It’s like 5-10 exceptions per day vs thousands of requests.

We can’t really see any side effects (other than our error tracker getting spammed), but as it explicitly states “Please report a bug.”, I’ll obey.

Note: We are using EF Core, thus also Npgsql.EntityFrameworkCore.PostgreSQL, but given the exception I guessed it still belongs in this project.

Exception message: Npgsql.NpgsqlException: Could not find free slot in Npgsql.NpgsqlConnector[] when closing. Please report a bug.
Stack trace:
   at Npgsql.ConnectorPool.CloseConnector(NpgsqlConnector connector)
   at Npgsql.NpgsqlConnection.<>c__DisplayClass87_0.<<Close>g__CloseAsync|0>d.MoveNext()
--- End of stack trace from previous location ---
   at Npgsql.NpgsqlConnection.DisposeAsync()
   at Microsoft.EntityFrameworkCore.Storage.RelationalConnection.DisposeAsync()
   at Microsoft.EntityFrameworkCore.DbContext.Microsoft.EntityFrameworkCore.Infrastructure.IResettableService.ResetStateAsync(CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.Internal.DbContextPool`1.ReturnAsync(IDbContextPoolable context, CancellationToken cancellationToken)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.<DisposeAsync>g__Await|15_0(Int32 i, ValueTask vt, List`1 toDispose)
   at Microsoft.AspNetCore.Http.Features.RequestServicesFeature.<DisposeAsync>g__Awaited|9_0(RequestServicesFeature servicesFeature, ValueTask vt)
   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.<FireOnCompleted>g__ProcessEvents|227_0(HttpProtocol protocol, Stack`1 events)

Further technical details

Npgsql version: 5.0.7 (but also appeared in previous versions, at least since two months) PostgreSQL versions: 12.4 and 13.3 (depending on the system) Operating systems: Linux, x86 and ARM

• v12.4 are Amazon Aurora instances (x86)
• v13.3 are Ubuntu 20.04 machines, including x86 and ARM (AWS Graviton 2) instances

Other details about my project setup:

• Applications run within docker containers
  • Docker 20.10
  • Docker image based on mcr.microsoft.com/dotnet/aspnet:5.0 (=Debian Buster)
• .NET5
• Services are ASP.NET Core applications with REST and/or gRPC interfaces
• EF Core 5.0.7
• Libs are usually up to date, at least for the projects where the error currently appears
• DI-Container-registration is just standard: services.AddDbContext<AppDbContext>(options => options.UseNpgsql(AppSettings.Database.ConnectionString, o => o.SetPostgresVersion(AppDbContext.PostgresVersion)))
• A DbContexts lifetime is limited to the request - it gets opened somewhen when processing a request and disposed once the request is complete (as you can also see in the stack). That happens pretty often. May this be a race condition?
• No special settings in the connection string, just host, user, pw, app name and ssl mode

[vonzshik]

Yep, I see it - the problem is with the TransactionScope’s rollback after a connection is broken. The test below reproduces this bug:

[Test]
[IssueLink("https://github.com/npgsql/npgsql/issues/3863")]
public void Bug3863([Values(true, false)] bool pooling)
{
	var csb = new NpgsqlConnectionStringBuilder(ConnectionStringEnlistOn)
	{
		Pooling = pooling
	};

	using (new TransactionScope())
	using (var conn = OpenConnection(csb))
	{
		conn.ExecuteNonQuery("SELECT 1");
		conn.Connector!.Break(new Exception("Some exception"));
	}
}

[jhartmann123]

And another one:

Npgsql.NpgsqlException: Could not find free slot in Npgsql.NpgsqlConnector[] when closing. Please report a bug.
  ?, in void ConnectorPool.CloseConnector(NpgsqlConnector connector)
  ?, in void VolatileResourceManager.Dispose()
  ?, in void VolatileResourceManager.Rollback(Enlistment enlistment)
  ?, in void VolatileEnlistmentAborting.EnterState(InternalEnlistment enlistment)
  ?, in void VolatileEnlistmentActive.InternalAborted(InternalEnlistment enlistment)
  ?, in void TransactionStateAborted.EnterState(InternalTransaction tx)
  ?, in void TransactionStateActive.Rollback(InternalTransaction tx, Exception e)
  ?, in void Transaction.Rollback()
  ?, in void TransactionScope.InternalDispose()
  ?, in void TransactionScope.Dispose()
  ?, in async Task<T> TransactionScopeHandler.RunInTransactionScope<T>(Func<Task<T>> action, TransactionScopeOption scopeOption)