Rate limiter causes thread pool exhaustion

When trying to load test the sliding window limiter I noticed that it quickly lead to redis timeouts especially in environments with limited cpu count and/or and a bit of latency to redis. This seems to be caused by exhausting the thread pool. Looking into the issue I found the following: The TryAcquireAsync function in RateLimitingMiddleware.cs first does a synchronous AttemptAcquire call before falling back to before falling back to AcquireAsync on the RateLimiter. This means unless something goes wrong, only synchronous StackExchange Redis calls are performed by this package.

Looking at the documentation on the RateLimiter class it says:

AcquireAsync(Int32, CancellationToken) Wait until the requested permits are available or permits can no longer be acquired.
AttemptAcquire(Int32) Fast synchronous attempt to acquire permits.

So this isn’t just the usual case of having an async and a blocking implementation but meant to be distinct functions that can both be used.

I have to admit that I am not sure I fully get the intent of the interface with regards to waiting until a permit is available. But I think the way to get proper scalability would probably be to never return a lease from the synchronous call and only reach out to redis in the asynchronous one.

Would this be an appropriate change? Bit surprised I am the first one to stumble over this so maybe I am on the wrong track completely.

[cristipufu]

We could overwrite these classes in the RedisRateLimiting.AspNetCore package 🤔

[kamilslusarczyk]

For reference: related issue created in MS’s repo: https://github.com/dotnet/runtime/issues/88592