RedisCommandTimeoutException when there is a network glitch

Hi,

We have a daemon process which pushes data to redis on AWS using lettuce-core. I have noticed that during network glitch i.e when a connection is disconnected , I get below exception :

io.lettuce.core.RedisCommandTimeoutException: Command timed out
at io.lettuce.core.LettuceFutures.awaitOrCancel(LettuceFutures.java:114)
at io.lettuce.core.FutureSyncInvocationHandler.handleInvocation(FutureSyncInvocationHandler.java:62)
at io.lettuce.core.internal.AbstractInvocationHandler.invoke(AbstractInvocationHandler.java:80)
at com.sun.proxy.$Proxy11.hmget(Unknown Source)

Code to reproduce :

public class TestMain {
	public static void main(String[] args) {
		Duration timeout = Duration.ofMillis(10000);
		RedisClient client = RedisClient.create(RedisURI.create("redis://127.0.0.1:6379"));
		client.setDefaultTimeout(timeout);
		RedisCommands<String, String> commands = client.connect().sync();
		String[] arr = { "test", "test2" };
		int i = 1;
		while (i <= 1000) {
			try {
				List<KeyValue<String, String>> op = commands.hmget("TEST", arr);
				System.out.println(op);
				i++;
			} catch (Exception e) {
				e.printStackTrace();
			}
		}
	}
}

Version :

<dependency>
	<groupId>io.lettuce</groupId>
	<artifactId>lettuce-core</artifactId>
	<version>5.0.3.RELEASE</version>
</dependency>

After the connection is re-established it works fine. The time out given is around 10 seconds and network connection is fixed before 10 seconds but still this exception occur. Ideally it should have waited for 10 seconds before throwing the exception.

Also the exception thrown seems to be incorrect i.e RedisCommandTimeoutException , could have been something different.

[jaimess]

this params would mend it: echo “1” > /proc/sys/net/ipv4/tcp_keepalive_intvl echo “10” > /proc/sys/net/ipv4/tcp_keepalive_probes echo “6” > /proc/sys/net/ipv4/tcp_keepalive_time echo “3” > /proc/sys/net/ipv4/tcp_retries2

mp911de is right but it may worth have a look to cassandra or elasticsearch drivers which are able to recover from these problem

[jaimess]

diving a bit I found nettycustommizer, this works too:

    NettyCustomizer nettyCustomizer = new NettyCustomizer() {
        
        @Override
        public void afterChannelInitialized(Channel channel) {
            channel.pipeline().addLast(
                    new IdleStateHandler(readerIdleTimeSeconds, writerIdleTimeSeconds, allIdleTimeSeconds));
            channel.pipeline().addLast(new ChannelDuplexHandler() {
                @Override
                public void userEventTriggered(ChannelHandlerContext ctx, Object evt) throws Exception {
                    if (evt instanceof IdleStateEvent) {
                        ctx.disconnect();
                    }
                }
            });
        }
        
        @Override
        public void afterBootstrapInitialized(Bootstrap bootstrap) {
        }
        
    };
    ClientResources res = DefaultClientResources.builder()
            .nettyCustomizer(nettyCustomizer ).build();