lettuce cluster multiGet RedisCommandTimeoutException cause memory leak
See original GitHub issueI used lettuce 5.0.5.RELEASE and Spring Boot 2.0
and I write the code to multiGet value
@Autowired
private RedisTemplate<String, byte[]> redisTemplate;
redisTemplate.opsForValue().multiGet(
ids.stream()
.map(RedisVideoProfileCache::genCacheKey)
.collect(Collectors.toSet())
)
Whenever I start the service for a long time, some app’s memory is leaked I download a jmap to debug it
num #instances #bytes class name
----------------------------------------------
1: 5668369 3397294608 [B
2: 9985354 455769512 [Ljava.lang.Object;
3: 9797176 313509632 java.util.HashMap$Node
4: 9700404 232809696 java.util.concurrent.CompletableFuture
5: 8891675 213400200 java.util.ArrayList
6: 3189943 127597720 io.lettuce.core.output.KeyValueListOutput
7: 2802968 112118720 java.util.concurrent.CompletableFuture$BiRelay
8: 3190015 102080480 io.lettuce.core.protocol.AsyncCommand
9: 3190015 102080480 io.lettuce.core.protocol.Command
10: 2358668 94346720 java.util.concurrent.CompletableFuture$UniWhenComplete
11: 1077001 92868160 [C
12: 3218728 77249472 io.lettuce.core.protocol.CommandArgs$KeyArgument
13: 2399019 76768608 java.util.ArrayList$Itr
14: 3190015 76560360 io.lettuce.core.protocol.CommandArgs
15: 107450 76403064 [Ljava.util.HashMap$Node;
16: 1572519 62900760 java.util.concurrent.CompletableFuture$UniApply
17: 2411845 57884280 io.lettuce.core.KeyValue
18: 3224435 51590960 java.lang.Integer
19: 801185 32047400 io.lettuce.core.cluster.ClusterCommand
20: 786213 31448520 java.util.concurrent.CompletableFuture$UniHandle
21: 1251582 30037968 java.util.concurrent.CompletableFuture$CoCompletion
22: 225886 29931880 [I
23: 457000 25592000 org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation
24: 786283 25161056 io.lettuce.core.DefaultConnectionFuture
25: 826014 23765088 [Ljava.util.concurrent.CompletableFuture;
26: 853506 20484144 java.lang.String
27: 786652 18879648 io.lettuce.core.cluster.PooledClusterConnectionProvider$$Lambda$1028/1326396018
28: 786652 18879648 io.lettuce.core.cluster.PooledClusterConnectionProvider$$Lambda$1030/1117636922
29: 786652 18879648 io.lettuce.core.cluster.PooledClusterConnectionProvider$$Lambda$1031/481259366
the byte[] used large than normal case
When I checked this app error log, I found large count RedisCommandTimeoutException
less c67003c59bbb854f176124c48d4c172a0d7aff1b1d14725705a7d3e57aad9e4a.log | grep ERROR | grep "io.lettuce.core.RedisCommandTimeoutException: Command timed out" | wc
12804 320100 3111372
less c67003c59bbb854f176124c48d4c172a0d7aff1b1d14725705a7d3e57aad9e4a.log | grep ERROR | grep -v "io.lettuce.core.RedisCommandTimeoutException: Command timed out" | wc
20 467 4835
I suspect this exception caused a memory leak, why ?
- Reference documentation: https://lettuce.io/docs/
- Wiki: https://github.com/lettuce-io/lettuce-core/wiki
- Javadoc: https://lettuce.io/core/release/api/
- Gitter Chat: https://gitter.im/lettuce-io/Lobby
- Mailing list https://groups.google.com/d/forum/lettuce-redis-client-users or lettuce-redis-client-users@googlegroups.com
- StackOverflow: https://stackoverflow.com/questions/tagged/lettuce
- Twitter: If it’s just a quick question you can ping our Twitter: https://twitter.com/LettuceDriver
- Also have a look at the readme for more information on how to get support: https://github.com/lettuce-io/lettuce-core/blob/master/README.md
Issue Analytics
- State:
- Created 5 years ago
- Comments:19 (4 by maintainers)
Top Results From Across the Web
Memory leak Error when connect redis with lettuce client
This type of error typically occurs when multiple instances of RedisClient get created without sharing ClientResources .
Read more >lettuce-io/Lobby - Gitter
I have a problem getting a connection for Cluster working. ... I'm trying to identify the root cause of a memory leak in...
Read more >Lettuce Reference Guide
Reactor based applications can sustain very high throughput message rates and operate with a very low memory footprint, making it suitable ...
Read more >[Java] New Relic agent leaking memory when Lettuce Redis ...
Newrelic agent is leaking memory in our java app which is using redis reactive library(Lettuce) and became major blocker to go 100 %...
Read more >Memory leak in ray head
CPU and memory usage on ray-head pod is still increasing and has to be restarted ... objects in cluster but it is probably...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
Watch out for the channel handlers (in particular
CommandHandler
) and the attachedDefaultEndpoint
. It’s interesting to inspect the state of the connection, whether it’s connected and how many instances of connections you have. If you don’t have any concerns regarding data privacy issues with your heap dump, feel free to share it with me.Upgrade lettuce version to 5.1.0.RELEASE fixed.