Memory leak in IdleConnectionWatcher
See original GitHub issueHey,
I have a production environment which uses a load balancer over several master Redis instances, so the actual IPs of the Redis instances switch with every request.
I update Redisson from some 3.3.* version to 3.15.3 and started to notice odd memory consumption: something keeps piling up in the Old Generation and never gets garbage-collected. Normally, it would take about a day to reach Old Gen mem cap and in a few days, so when other objects make it to Old Gen, the app just dies with OOM.
I dumped the memory, it looks like when IPs change, new ClientConnectionsEntry
items are allocated and attached the linked queue in the IdleConnectionWatcher
which is never pruned, unless the client is shut down. I can’t tell how many nodes are in those queues, probably too many, because IdleConnectionWatcher
is about 30Mb big.
Unfortunately, I cannot change the infrastructure I use. Any chance there’s any config that could just mitigate it for now?
Expected behavior Application does not run out of memory if connection entries are re-created frequently and dangling objects are cleaned up.
Actual behavior Strong references to previously created objects are retained and keep piling up until the app gets OOM-killed.
Steps to reproduce or test case Tricky: Redis instances should be behind a load balancer, which should have a resolvable hostname. The idea is to make Redisson try to rebuild the connection pool every time it makes a request.
Redis version Idk, because I don’t have access to the infrastructure.
Redisson version 3.15.3
Redisson configuration
- Clients connect to Master nodes.
- Clients are single-threaded.
- Clients attempt to resolve hostname to connect to actual Redis instances.
Issue Analytics
- State:
- Created 2 years ago
- Comments:9 (5 by maintainers)
Here it is:
redisson-3.15.5-SNAPSHOT.jar.zip
Awesome, thanks! I wonder if there is a snapshot artifact I could try out, there some style check failing on
master
, I can’t build it myself.