Redisson indefinitely try to reconnect on old sentinel instance

We using redisson for our tomcat session cache in Kubernetes cluster. We have setup a sentinel cluster with 3 redis instance.

We have some issue when one of our redis pod changes. This can happen when maintaining a node. In this case the redis pod is recreated on another node but with a new IP.

For connect to sentinel we use a statefulset headless service.

Expected behavior

Redisson reconnect to the new redis node and stop communicating with the old pod

Actual behavior

Redisson reconnect to the new redis node BUT continue to try reconnect on old node indefinitely.

Steps to reproduce or test case

1. Setup redis sentinel cluster in Kubernetes (with statefulset). Our pod have 2 container, one for redis (port 6379), and one for is sentinel (port 26379).
2. Start tomcat pod with redisson config for session.
3. kubectl delete pod <redis slave pod>

Redis version

6.0.12

Redisson version

3.15.3

Redisson configuration

sentinelServersConfig:
  password: ${REDIS_PASSWORD}
  subscriptionConnectionMinimumIdleSize: 1
  subscriptionConnectionPoolSize: 10
  slaveConnectionMinimumIdleSize: 2
  slaveConnectionPoolSize: 10
  masterConnectionMinimumIdleSize: 5
  masterConnectionPoolSize: 10
  idleConnectionTimeout: 5000
  pingConnectionInterval: 0
  dnsMonitoringInterval: 5000
  checkSentinelsList: true
  clientName: ${HOSTNAME}
  sentinelAddresses:
  - "redis://redis-${ENV}-rds01-0.redis-${ENV}-rds01-headless.infra.svc.cluster.local:26379"
  - "redis://redis-${ENV}-rds01-1.redis-${ENV}-rds01-headless.infra.svc.cluster.local:26379"
  - "redis://redis-${ENV}-rds01-2.redis-${ENV}-rds01-headless.infra.svc.cluster.local:26379"
  masterName: "redisk8s"

[ralmn]

I confirm that it is corrected. Thanks 😃

[kingkonghong]

It’s difficult to reproduce the problem. Now, this happens only when the entire redis cluster goes down and returns to normal again. And I collected some TRACE level logs. redisson.log