How to timeout when Sentinel Master is unreachable
See original GitHub issueIs there a way to add a timeout to a get/set action? I’m looking for this in the docs and it’s either not there, or I’m somehow not seeing it.
I’m connecting ioredis
to 3 sentinels using a master/slave setup.
I can trigger a failover manually and everything works perfectly on the next request.
But, when I tell the current master to simulate an “unreachable” status:
redis-cli -p 6379 DEBUG sleep 30
- the sentinels notice and elect a new master
- but the app waits for 30sec until the master is reachable again (now as a slave), then reconnects to the new sentinel master
My config is
{
sentinels: [...the 3 sentinels],
name: "mymaster",
reconnectOnError: err => /^READONLY/.test(err.message),
}
Issue Analytics
- State:
- Created 6 years ago
- Reactions:2
- Comments:5
Top Results From Across the Web
High availability with Redis Sentinel
If two Sentinels agree at the same time about the master being unreachable, one of the two will try to start a failover....
Read more >Configuring Redis servers for failover operation
If Redis. sentinel is configured and the master server is down, then one of the slave servers will be reconfigured as the master....
Read more >3.4 Exercise - Sentinel Hands-on - Redis Developer Hub
sentinel failover-timeout - if a Sentinel voted another Sentinel for the failover of a given master, it will wait this many milliseconds to...
Read more >Redis Sentinel Documentation
This delay is the failover-timeout you can configure in sentinel.conf . This means that Sentinels will not try to failover the same master...
Read more >https://docs.sensu.io/sensu-core/latest/files/sent...
protected-mode no daemonize yes pidfile /var/run/redis/redis-sentinel.pid ... the master. sentinel parallel-syncs mymaster 1 # sentinel failover-timeout ...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
Also, here’s the Redis docs that show that
sleep
is a good way to simulate a master hanging. https://redis.io/topics/sentinel#testing-the-failoverSo I have also followed described scenario and in my opinion this bug needs to be reopened. My setup: 3 sentinels, 3 redis slaves, 1 redis master. Sentinels env_vars: “REDIS_SENTINEL_DOWN_AFTER_MILLISECONDS”: “10000”. I am simulating Redis master down with command: redis-cli -p 6379 DEBUG sleep 60. Whats going on is: After 10 seconds, sentinels know right a way about the failure and elects new master. ioredis does NOT detect that! No error is passed. ioredis is still connected to the old master and my app hangs… After 60 seconds when its back, ioredis detects that old master is not a master anymore and tries to reconnect to the new one with a success. The question is: why ioredis does not get new master from sentinels?