Kestrel memory pool does not reliably decrease with load

Describe the bug

While load testing a new service using SignalR, we noticed that memory usage does not always decrease after connections drop off. Sometimes it decreases a minute or two after the connections end, but sometimes it just never recovers. This causes us two problems:

1. When an instance nears its memory limit, it will reject new connections (achieved using custom middleware that responds with a 503) in order to protect its existing connections. But since memory usage does not decrease, the instance will never accept new connections, even after most of its existing connections end
2. Our service autoscales on memory, but because memory usage stays high our service will rarely scale-down when under low load. This is less of a problem, it just costs a bit to run with extra capacity

This shows memory usage increasing with number of connections (tracked using the OnConnectedAsync/OnDisconnectedAsync methods of our Hub), but not reliably decreasing afterwards:

The service is running in an Alpine 3.12 docker container, but we notice similar behaviour when running locally in Windows 10. We initially assumed this was a simple memory leak, but when making a large number of connections over time while keeping the number of concurrent connections low, memory usage does not increase.

When taking a heap dump, we see memory fairly dominated by Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets!Buffers.MemoryPoolBlock:

To Reproduce

1. Make a large number of concurrent connections to a SignalR Core service
2. End connections
3. Monitor memory - it should decrease, but usually does not

Further technical details

• ASP.NET Core version 3.1.6, using docker image based on mcr.microsoft.com/dotnet/core/aspnet:3.1.6-alpine3.12
• Built with SDK 3.1.302, using docker image based on mcr.microsoft.com/dotnet/core/sdk:3.1.302-alpine3.12
• Reproduced locally on Windows 10 using ASP.NET Core 3.1.6, SDK 3.1.202

[BrennanConroy]

Kestrel will pool internal buffer up to a limit and will not let them go. We should consider having a settable limit or release strategy for the pool.

cc @halter73

[adityamandaleeka]

Closing this as a duplicate of https://github.com/dotnet/aspnetcore/issues/27394