question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Allow providing custom ClusterTopologyRefresh implementation.

See original GitHub issue

Feature Request

Is your feature request related to a problem? Please describe

During our load tests we have noticed an increasing level of CPU in each node of our Redis cluster that was proportional to the number of server connected.

Each server was on idle (no requests) and had a single Redis lettuce client instance running.

By looking at the Redis metrics we noticed that each node was constantly accepting new connections and reading the command logs we narrowed down the cause to the cluster topology refresh requests.

The DefaultClusterTopologyRefresh class creates new connection every time is called.

We have already increased the cluster topology refresh and we have switched the dynamicRefreshSources off, so the client refresh the topology is done against the seeds, in our case a single node.

With this adjustments we saw improvements but the CPU problem is still there.

This limits our ability to scale the number of server instances.

Describe the solution you’d like

Modify the DefaultClusterTopologyRefresh class to keep the connection alive so they can be re-used between topology refresh calls.

Maybe the ConnectionTracker class can be leveraged to store the connections between the calls.

Teachability, Documentation, Adoption, Migration Strategy

I will be happy to implement this feature, but I would like to know your opinion and maybe suggestions around the implementation.

Thanks

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:5 (5 by maintainers)

github_iconTop GitHub Comments

1reaction
mp911decommented, Jan 22, 2021

That’s in place now.

0reactions
alessandrosimi-sacommented, Jan 22, 2021

Nice, thanks a lot!

Read more comments on GitHub >

github_iconTop Results From Across the Web

Lettuce Reference Guide
Allow randomization of read candidates using Redis Cluster. SSL support for Redis Sentinel. 2.6. What's new in Lettuce 5.1. Add support for ...
Read more >
lettuce-io/Lobby - Gitter
You need to enable topology refresh so Lettuce can obtain the new cluster view via CLUSTER NODES . Make sure that your seed...
Read more >
Lettuce client for Redis - Cluster Topology Refresh Options not ...
Adjusting the connection timeout will give you the desired result. ex) redisURIs.add(RedisURI.create("redis://127.0.0.1:7000/0?timeout=10s"));.
Read more >
Amazon Elasticache FAQs - Amazon Web Service
For example, a cluster consisting of ten shards of 6 GB each will provide 60 GB ... With Amazon VPC, you can define...
Read more >
Redis Cluster topology - VMware Docs
Providing the values above, the cluster will have 3 masters and, each master, will have 2 replicas. NOTE: By default cluster.init will be...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found