question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Memory leak when using cluster mode to read by RBatch

See original GitHub issue

Expected behavior No memory leak Actual behavior Memory leak Steps to reproduce or test case `public class CNRedisHandleFactory { public static RedissonClient redisson; public static BatchOptions options; public static void main(String[] args) throws InterruptedException { // String clusterAddress = “192.168.11.73:7000||192.168.11.73:7001||192.168.11.83:7000||192.168.11.83:7001||192.168.11.93:7000||192.168.11.93:7001||192.168.11.107:7000||192.168.11.107:7001||192.168.11.111:7000||192.168.11.111:7001||192.168.11.121:7000||192.168.11.121:7001||192.168.11.131:7000||192.168.11.131:7001”; String ipAddress = “10.253.0.41:7000||10.253.0.41:7001||10.253.0.87:7000||10.253.0.87:7001||10.253.0.112:7000||10.253.0.112:7001||10.253.0.138:7000||10.253.0.138:7001||10.253.0.142:7000||10.253.0.142:7001||10.253.0.159:7000||10.253.0.159:7001||10.253.0.192:7000||10.253.0.192:7001”; int timeOut = 5000;

    List<String> sp = Arrays.asList(ipAddress.split("\\|\\|"));
    List<String> stringList = sp.stream().map(x->"redis://"+x).collect(Collectors.toList());
    String[] ip=stringList.toArray(new String[stringList.size()]);
    Config config = new Config();
    config.useClusterServers().addNodeAddress(ip)
            .setScanInterval(1000).setReadMode(ReadMode.MASTER_SLAVE)
            .setTimeout(timeOut)
            .setRetryAttempts(0)
            .setMasterConnectionPoolSize(64)
            .setSlaveConnectionPoolSize(64)
            .setMasterConnectionMinimumIdleSize(24)
            .setSlaveConnectionMinimumIdleSize(24)
    ;

    options = BatchOptions.defaults()
            .responseTimeout(timeOut, TimeUnit.MILLISECONDS)
            .retryInterval(1500, TimeUnit.MILLISECONDS)
            .retryAttempts(0);

    redisson = Redisson.create(config);

    int thread = Integer.parseInt(args[0]);
    int sleep = Integer.parseInt(args[1]);
    System.out.println("thread:" + thread + " sleep:" + sleep);

    for (int i = 0; i < thread; i++) {
        new Thread(new Runnable() {
            @Override
            public void run() {
                while (true) {
                    try {
                        Set<String> set = new HashSet();
                        int randomKeyNum = new Random().nextInt(10);
                        for (int loop = 0; loop <= randomKeyNum; ++loop) {
                            set.add(redisson.getKeys().randomKey());
                        }
                        Map<String, byte[]> temp = getByteArrayMap(set.toArray(new String[set.size()]));

                    } catch (Exception e) {
                        e.printStackTrace();
                    }
                    try {
                        Thread.sleep(sleep);
                    } catch (InterruptedException e) {
                        e.printStackTrace();
                    }
                }
            }
        }).start();
    }

    Thread.sleep(1000000000000L);
}

public static Map<String,byte[]> getByteArrayMap(String... keys){
    Map<String, byte[]> res = new HashMap<>();
    RBatch batch = redisson.createBatch(options);
    for (String key : keys) {
        batch.getBucket(key, new ByteArrayCodec()).getAsync();
    }
    List<?> res1 = batch.execute().getResponses();
    for (int i = 0; i < res1.size(); i++) {
        if (res1.get(i) != null) {
            res.put(keys[i], (byte[]) res1.get(i));
        }
    }
    return res;
}

} ` Redis version 5.0 and cluster with 7 nodes Redisson version 3.12.2 JDK 1.8.0_91 Redisson configuration

Redisson configuration
		config.useClusterServers().addNodeAddress(ip)
        .setScanInterval(1000).setReadMode(ReadMode.MASTER_SLAVE)
        .setTimeout(timeOut)
        .setRetryAttempts(0)
        .setMasterConnectionPoolSize(64)
        .setSlaveConnectionPoolSize(64)
        .setMasterConnectionMinimumIdleSize(24)
        .setSlaveConnectionMinimumIdleSize(24)
;

options = BatchOptions.defaults()
        .responseTimeout(timeOut, TimeUnit.MILLISECONDS)
        .retryInterval(1500, TimeUnit.MILLISECONDS)
        .retryAttempts(0);

and request controller, 500 threads and each HTTP GET request will fetch 1-10 RandomKeys from Redis cluster in batch mode then sleep 30ms. JVM PARAMS JAVA_OPTS="-Xms400M -Xmx400M -XX:MaxDirectMemorySize=1G "

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Comments:13 (6 by maintainers)

github_iconTop GitHub Comments

1reaction
mrnikocommented, Feb 20, 2020

use new FstCodec(FSTConfiguration.createDefaultConfiguration(), false) as codec or try Kryo5Codec

0reactions
wuwangbencommented, Feb 22, 2020

use new FstCodec(FSTConfiguration.createDefaultConfiguration(), false) as codec or try Kryo5Codec Sorry, Wrong verification before. Use new FstCodec(FSTConfiguration.createDefaultConfiguration(), false) This problem did not reproduce. This problem did not reproduce,Thank you.

Read more comments on GitHub >

github_iconTop Results From Across the Web

A step-by-step guide for debugging memory leaks in Spark ...
A step-by-step guide for debugging memory leaks in Spark Applications. Just a bit of context. We at Disney Streaming Services use Apache Spark ......
Read more >
mrniko/redisson - Gitter
@mrniko We are facing a memory leak issue caused by FSTCodec used by the ... @mrniko Is there an example of redisson usage...
Read more >
Redisson/CHANGELOG and Redisson Releases (Page 5)
Redisson - Redis Java client with features of In-Memory Data Grid. ... method of Spring Data Redis isn't compatible with Redis cluster mode...
Read more >
A Guide to Redis with Redisson - Baeldung
A quick and practical guide to using Redisson for accessing Redis from a Java application.
Read more >
pm2 constantly leaking memory when running app in cluster ...
OP says they're using cluster mode. I'd imagine pm2 only kills the one process that's leaky in that case, and the rest can...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found