Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

RedisException: ERR Error running script while using RRateLimiter

See original GitHub issue

I currently have an application where I first create a RRateLimiter. I then acquire one permit in a loop and do an operation. This application is intended to run in a distributed fashion. To test the behavior, I set off concurrent invocations of this program. While it works well for the most part, it does throw an error occasionally as described below.

Expected behavior No error. The thread waits to acquire a permit and does so.

Actual behavior Occasionally, the limiter.acquire() in the below code throws the following error - org.redisson.client.RedisException: ERR Error running script (call to f_df5536206d0c440727a5f17ab38851a31af682c7): @user_script:1: user_script:1: bad argument #2 to 'unpack' (string expected, got nil) . channel: [id: 0x21ed2b8c, L:/169.254.76.1:60060 - R:/10.24.141.126:6379] command: (EVAL), params: [local rate = redis.call('hget', KEYS[1], 'rate');local interval = redis.call('hget', KEYS[1], 'inter..., 5, test, {test}:value, {test}:value:a070c033-9d81-4fd9-b2f7-55a15a4cc468, {test}:permits, {test}:permits:a070c033-9d81-4fd9-b2f7-55a15a4cc468, 1, 1597793996430, -2193289253268890349]

Steps to reproduce or test case Code snippet in AWS lambda talking to elasticache -

RRateLimiter limiter = redisson.getRateLimiter("test");
limiter.trySetRate(RateType.OVERALL, 1, 2, RateIntervalUnit.SECONDS);
while (true) {
                try {
                    limiter.acquire();
                    context.getLogger().log("get semaphore");
                    sqs.sendMessage(new SendMessageRequest()
                            .withQueueUrl("https://sqs_url")
                            .withMessageBody(new Date().toString()));
                    context.getLogger().log("sent to sqs");
                }
                catch (Exception ex) {
                    context.getLogger().log("Error in loop:" + ex);
                }
            }

Redis version Elasticache, 5.0.6

Redisson version 3.13.3

Redisson configuration Elasticache cluster config

Issue Analytics

State:
Created 3 years ago
Comments:32 (13 by maintainers)

Top GitHub Comments

1reaction

mrnikocommented, Sep 2, 2020

fixed in https://github.com/redisson/redisson/issues/3025

0reactions

adaboinanagarrocommented, Nov 17, 2022

Facing same issue, please find below code i am using and the issue i am facing

ApplicationRunner throttlingInitializer() { return args -> { log.info(“Rate limits: {} {} {} {}”, kv(“client”, clientDefaultRequestRateLimit), kv(“incoming”, incomingRequestRateLimit), kv(“lbs”, lbsRequestRateLimit), kv(“unit”, rateLimitUnit));

        // Clear rate limiters to ensure rate limit updates take effect

        log.debug("Resetting incoming rate limiter");
        redisson.getRateLimiter(INCOMING_RATE_LIMITER_NAME).delete();

        // TODO: Re-evaluate this. Though it is a simple solution to ensure client rate limits are updated, it
        //  may not be a scalable one.
        clientService.listClients().subscribe(client -> {
            log.debug("Resetting client rate limiter for {}", client.getClientId());
            redisson.getRateLimiter(CLIENT_RATE_LIMITER_NAME_PREFIX + client.getClientId()).delete();
        });

        log.debug("Resetting LBS rate limiter");
        redisson.getRateLimiter(SERVICE_RATE_LIMITER_NAME_PREFIX + SERVICE_ID_LBS).delete();
    };

{“@timestamp”:“2022-10-26T18:10:18.862Z”,“@version”:“1”,“message”:“[35f54d06-1] 500 Server Error for HTTP POST "/mlp/"”,“logger_name”:“org.springframework.boot.autoconfigure.web.reactive.error.AbstractErrorWebExceptionHandler”,“thread_name”:“redisson-netty-8-9”,“level”:“ERROR”,“level_value”:40000,“stack_trace”:“org.redisson.client.RedisException: ERR Error running script (call to f_a23369f7eb133a5016039b245b3fd2482bcafcc3): @user_script:1: user_script:1: attempt to perform arithmetic on a nil value. channel: [id: 0x0e54249e, L:/10.128.82.240:37290 - R:redis-11111.internal.c16265.us-east-1-mz.ec2.cloud.rlrcp.com/10.0.0.39:11111] command: (EVAL), promise: RedissonPromise [promise=ImmediateEventExecutor$ImmediatePromise@640d6897(incomplete)], params: [local rate = redis.call(‘hget’, KEYS[1], ‘rate’);local interval = redis.call(‘hget’, KEYS[1], 'inter…, 5, request_rate_limiter:client:12601245, {request_rate_limiter:client:12601245}:value, {request_rate_limiter:client:12601245}:value:e6c49a0c-2958-46d0-b3c8-8964628ed07a, {request_rate_limiter:client:12601245}:permits, {request_rate_limiter:client:12601245}:permits:e6c49a0c-2958-46d0-b3c8-8964628ed07a, 1, 1666807818861, [-73, 28, 44, 105, -92, -126, 100, -99]]\n\tat org.redisson.client.handler.CommandDecoder.decode(CommandDecoder.java:370)\n\tSuppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException: \nError has been observed at the following site(s):\n\t*_checkpoint ⇢ org.springframework.cloud.gateway.filter.WeightCalculatorWebFilter [DefaultWebFilterChain]\n\t*_checkpoint ⇢ org.springframework.security.web.server.authorization.AuthorizationWebFilter [DefaultWebFilterChain]\n\t*_checkpoint ⇢ org.springframework.security.web.server.authorization.ExceptionTranslationWebFilter [DefaultWebFilterChain]\n\t*_checkpoint ⇢ org.springframework.security.web.server.authentication.logout.LogoutWebFilter [DefaultWebFilterChain]\n\t*_checkpoint ⇢ org.springframework.security.web.server.savedrequest.ServerRequestCacheWebFilter [DefaultWebFilterChain]\n\t*_checkpoint ⇢ org.springframework.security.web.server.context.SecurityContextServerWebExchangeWebFilter [DefaultWebFilterChain]\n\t*_checkpoint ⇢ org.springframework.security.web.server.authentication.AuthenticationWebFilter [DefaultWebFilterChain]\n\t*_checkpoint ⇢ org.springframework.security.web.server.context.ReactorContextWebFilter [DefaultWebFilterChain]\n\t*_checkpoint ⇢ org.springframework.security.web.server.header.HttpHeaderWriterWebFilter [DefaultWebFilterChain]\n\t*_checkpoint ⇢ org.springframework.security.config.web.server.ServerHttpSecurity$ServerWebExchangeReactorContextWebFilter [DefaultWebFilterChain]\n\t*_checkpoint ⇢ org.springframework.security.web.server.WebFilterChainProxy [DefaultWebFilterChain]\n\t*_checkpoint ⇢ org.springframework.boot.actuate.metrics.web.reactive.server.MetricsWebFilter [DefaultWebFilterChain]\n\t*__checkpoint ⇢ HTTP POST "/mlp/" [ExceptionHandlingWebHandler]\nOriginal Stack Trace:\n\t\tat org.redisson.client.handler.CommandDecoder.decode(CommandDecoder.java:370)\n\t\tat org.redisson.client.handler.CommandDecoder.decodeCommand(CommandDecoder.java:198)\n\t\tat org.redisson.client.handler.CommandDecoder.decode(CommandDecoder.java:137)\n\t\tat org.redisson.client.handler.CommandDecoder.decode(CommandDecoder.java:113)\n\t\tat io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:507)\n\t\tat io.netty.handler.codec.ReplayingDecoder.callDecode(ReplayingDecoder.java:366)\n\t\tat io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276)\n\t\tat io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)\n\t\tat io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)\n\t\tat io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)\n\t\tat io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)\n\t\tat io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)\n\t\tat io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)\n\t\tat io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)\n\t\tat io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)\n\t\tat io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:722)\n\t\tat io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:658)\n\t\tat io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:584)\n\t\tat io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:496)\n\t\tat io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)\n\t\tat io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)\n\t\tat io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)\n\t\tat java.base/java.lang.Thread.run(Unknown Source)\n”} {“@timestamp”:“2022-10-26T18:09:53.853Z”,“@version”:“1”,“message”:“Uncaught exception”,“logger_name”:“com.verizon.msz.gateway.exception.GlobalExceptionHandler”,“thread_name”:“redisson-netty-8-29”,“level”:“ERROR”,“level_value”:40000,“stack_trace”:“org.redisson.client.RedisException: ERR Error running script (call to f_a23369f7eb133a5016039b245b3fd2482bcafcc3): @user_script:1: user_script:1: attempt to perform arithmetic on a nil value. channel: [id: 0xf9570d69, L:/10.128.82.240:37374 - R:redis-11111.internal.c16265.us-east-1-mz.ec2.cloud.rlrcp.com/10.0.0.39:11111] command: (EVAL), promise: RedissonPromise [promise=ImmediateEventExecutor$ImmediatePromise@249da0e8(incomplete)], params: [local rate = redis.call(‘hget’, KEYS[1], ‘rate’);local interval = redis.call(‘hget’, KEYS[1], 'inter…, 5, request_rate_limiter:incoming, {request_rate_limiter:incoming}:value, {request_rate_limiter:incoming}:value:e6c49a0c-2958-46d0-b3c8-8964628ed07a, {request_rate_limiter:incoming}:permits, {request_rate_limiter:incoming}:permits:e6c49a0c-2958-46d0-b3c8-8964628ed07a, 1, 1666807793852, [-82, -20, 86, 58, 5, -58, 63, -118]]\n\tat org.redisson.client.handler.CommandDecoder.decode(CommandDecoder.java:370)\n\tSuppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException: \nError has been observed at the following site(s):\n\t*_checkpoint ⇢ org.springframework.cloud.gateway.filter.WeightCalculatorWebFilter [DefaultWebFilterChain]\n\t*_checkpoint ⇢ org.springframework.security.web.server.authorization.AuthorizationWebFilter [DefaultWebFilterChain]\n\t*_checkpoint ⇢ org.springframework.security.web.server.authorization.ExceptionTranslationWebFilter [DefaultWebFilterChain]\n\t*_checkpoint ⇢ org.springframework.security.web.server.authentication.logout.LogoutWebFilter [DefaultWebFilterChain]\n\t*_checkpoint ⇢ org.springframework.security.web.server.savedrequest.ServerRequestCacheWebFilter [DefaultWebFilterChain]\n\t*_checkpoint ⇢ org.springframework.security.web.server.context.SecurityContextServerWebExchangeWebFilter [DefaultWebFilterChain]\n\t*_checkpoint ⇢ org.springframework.security.web.server.authentication.AuthenticationWebFilter [DefaultWebFilterChain]\n\t*_checkpoint ⇢ org.springframework.security.web.server.context.ReactorContextWebFilter [DefaultWebFilterChain]\n\t*_checkpoint ⇢ org.springframework.security.web.server.header.HttpHeaderWriterWebFilter [DefaultWebFilterChain]\n\t*_checkpoint ⇢ org.springframework.security.config.web.server.ServerHttpSecurity$ServerWebExchangeReactorContextWebFilter [DefaultWebFilterChain]\n\t*_checkpoint ⇢ org.springframework.security.web.server.WebFilterChainProxy [DefaultWebFilterChain]\n\t*_checkpoint ⇢ org.springframework.boot.actuate.metrics.web.reactive.server.MetricsWebFilter [DefaultWebFilterChain]\n\t*__checkpoint ⇢ HTTP POST "/mlp" [ExceptionHandlingWebHandler]\nOriginal Stack Trace:\n\t\tat org.redisson.client.handler.CommandDecoder.decode(CommandDecoder.java:370)\n\t\tat org.redisson.client.handler.CommandDecoder.decodeCommand(CommandDecoder.java:198)\n\t\tat org.redisson.client.handler.CommandDecoder.decode(CommandDecoder.java:137)\n\t\tat org.redisson.client.handler.CommandDecoder.decode(CommandDecoder.java:113)\n\t\tat io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:507)\n\t\tat io.netty.handler.codec.ReplayingDecoder.callDecode(ReplayingDecoder.java:366)\n\t\tat io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276)\n\t\tat io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)\n\t\tat io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)\n\t\tat io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)\n\t\tat io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)\n\t\tat io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)\n\t\tat io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)\n\t\tat io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)\n\t\tat io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)\n\t\tat io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:722)\n\t\tat io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:658)\n\t\tat io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:584)\n\t\tat io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:496)\n\t\tat io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)\n\t\tat io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)\n\t\tat io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)\n\t\tat java.base/java.lang.Thread.run(Unknown Source)\n”} {“@timestamp”:“2022-10-26T18:09:53.853Z”,“@version”:“1”,“message”:“Uncaught exception”,“logger_name”:“com.verizon.msz.gateway.service.LoggingService”,“thread_name”:“redisson-netty-8-29”,“level”:“ERROR”,“level_value”:40000,“stack_trace”:“org.redisson.client.RedisException: ERR Error running script (call to f_a23369f7eb133a5016039b245b3fd2482bcafcc3): @user_script:1: user_script:1: attempt to perform arithmetic on a nil value. channel: [id: 0xf9570d69, L:/10.128.82.240:37374 - R:redis-11111.internal.c16265.us-east-1-mz.ec2.cloud.rlrcp.com/10.0.0.39:11111] command: (EVAL), promise: RedissonPromise [promise=ImmediateEventExecutor$ImmediatePromise@249da0e8(incomplete)], params: [local rate = redis.call(‘hget’, KEYS[1], ‘rate’);local interval = redis.call(‘hget’, KEYS[1], 'inter…, 5, request_rate_limiter:incoming, {request_rate_limiter:incoming}:value, {request_rate_limiter:incoming}:value:e6c49a0c-2958-46d0-b3c8-8964628ed07a, {request_rate_limiter:incoming}:permits, {request_rate_limiter:incoming}:permits:e6c49a0c-2958-46d0-b3c8-8964628ed07a, 1, 1666807793852, [-82, -20, 86, 58, 5, -58, 63, -118]]\n\tat org.redisson.client.handler.CommandDecoder.decode(CommandDecoder.java:370)\n\tSuppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException: \nError has been observed at the following site(s):\n\t*_checkpoint ⇢ org.springframework.cloud.gateway.filter.WeightCalculatorWebFilter [DefaultWebFilterChain]\n\t*_checkpoint ⇢ org.springframework.security.web.server.authorization.AuthorizationWebFilter [DefaultWebFilterChain]\n\t*_checkpoint ⇢ org.springframework.security.web.server.authorization.ExceptionTranslationWebFilter [DefaultWebFilterChain]\n\t*_checkpoint ⇢ org.springframework.security.web.server.authentication.logout.LogoutWebFilter [DefaultWebFilterChain]\n\t*_checkpoint ⇢ org.springframework.security.web.server.savedrequest.ServerRequestCacheWebFilter [DefaultWebFilterChain]\n\t*_checkpoint ⇢ org.springframework.security.web.server.context.SecurityContextServerWebExchangeWebFilter [DefaultWebFilterChain]\n\t*_checkpoint ⇢ org.springframework.security.web.server.authentication.AuthenticationWebFilter [DefaultWebFilterChain]\n\t*_checkpoint ⇢ org.springframework.security.web.server.context.ReactorContextWebFilter [DefaultWebFilterChain]\n\t*_checkpoint ⇢ org.springframework.security.web.server.header.HttpHeaderWriterWebFilter [DefaultWebFilterChain]\n\t*_checkpoint ⇢ org.springframework.security.config.web.server.ServerHttpSecurity$ServerWebExchangeReactorContextWebFilter [DefaultWebFilterChain]\n\t*_checkpoint ⇢ org.springframework.security.web.server.WebFilterChainProxy [DefaultWebFilterChain]\n\t*_checkpoint ⇢ org.springframework.boot.actuate.metrics.web.reactive.server.MetricsWebFilter [DefaultWebFilterChain]\n\t*__checkpoint ⇢ HTTP POST "/mlp" [ExceptionHandlingWebHandler]\nOriginal Stack Trace:\n\t\tat org.redisson.client.handler.CommandDecoder.decode(CommandDecoder.java:370)\n\t\tat org.redisson.client.handler.CommandDecoder.decodeCommand(CommandDecoder.java:198)\n\t\tat org.redisson.client.handler.CommandDecoder.decode(CommandDecoder.java:137)\n\t\tat org.redisson.client.handler.CommandDecoder.decode(CommandDecoder.java:113)\n\t\tat io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:507)\n\t\tat io.netty.handler.codec.ReplayingDecoder.callDecode(ReplayingDecoder.java:366)\n\t\tat io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276)\n\t\tat io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)\n\t\tat io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)\n\t\tat io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)\n\t\tat io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)\n\t\tat io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)\n\t\tat io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)\n\t\tat io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)\n\t\tat io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)\n\t\tat io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:722)\n\t\tat io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:658)\n\t\tat io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:584)\n\t\tat io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:496)\n\t\tat io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)\n\t\tat io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)\n\t\tat io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)\n\t\tat java.base/java.lang.Thread.run(Unknown Source)\n”,“error_message”:“Uncaught exception”,“error_type”:“SYSTEM_FAILURE”,“event”:“SYSTEM_ERROR”}