RateLimiter Question/Enhancement: Reduce memory footprint of high cardinality keys

I’d like to rate limit access to an operation that’s relatively expensive. My intention is to prevent some customers from over-utilizing my application since that could potentially result in degraded performance for other customers (multi-tenant application).

My application creates a wide key space for each of my customer; about 100 keys on average. I’d like to rate limit on these customer keys rather than on the customer identifier itself. If I have 5,000 customers using my application, this could translate to 5,000 * 100 = 500,000 Rate limiter keys.

Since stale/unused entries in RateLimiter registry aren’t cleaned up automatically (please correct my if I’m wrong), the memory footprint becomes significant.

I understand that I can override RateLimiterRegistry and perform cleanup on my side, but I feel this is not an uncommon use-case for a Rate Limiter and would like to get your thoughts on introducing 2 new config keys:

1. expiryTime - determines the duration after which each unused rate limiter key will be marked as stale.
2. cleanupInterval - determines the rate at which stale keys are cleaned up.

[RobWin]

Thank you for this PR

[asarkar]

@RobWin I see, so the idea is to use many rate limiters (presumably one per user), but have them “garbage collected”, if you may, after expiration. Now it makes sense to me.