Lightrun
question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Many exceptions and errors after sentinel failover with redisson 3.6.1

See original GitHub issue

Hello, I’m re-opening the issue #1272 for me it’s not fixed in 3.6.1, I get lot of errors long time after a fail-over.

redissonerrors

My setup :

The logs :

[2018-02-27 08:06:19,646] [WARN] [i.n.u.i.l.Slf4JLogger] [Null|127.0.0.1|Null|Null] - An exception was thrown by org.redisson.command.CommandAsyncService$15.operationComplete()
java.lang.NullPointerException: null
[2018-02-27 08:06:19,832] [WARN] [i.n.u.i.l.Slf4JLogger] [Null|127.0.0.1|Null|Null] - An exception was thrown by org.redisson.command.CommandAsyncService$15.operationComplete()
java.lang.NullPointerException: null
[2018-02-27 08:06:19,834] [WARN] [i.n.u.i.l.Slf4JLogger] [Null|127.0.0.1|Null|Null] - An exception was thrown by org.redisson.command.CommandAsyncService$15.operationComplete()
java.lang.NullPointerException: null
[2018-02-27 08:06:20,031] [WARN] [i.n.u.i.l.Slf4JLogger] [Null|127.0.0.1|Null|Null] - An exception was thrown by org.redisson.command.CommandAsyncService$15.operationComplete()
java.lang.NullPointerException: null
[2018-02-27 08:06:20,035] [WARN] [i.n.u.i.l.Slf4JLogger] [Null|127.0.0.1|Null|Null] - An exception was thrown by org.redisson.command.CommandAsyncService$15.operationComplete()
java.lang.NullPointerException: null
[2018-02-27 08:06:20,133] [WARN] [i.n.u.i.l.Slf4JLogger] [Null|127.0.0.1|Null|Null] - An exception was thrown by org.redisson.command.CommandAsyncService$15.operationComplete()
java.lang.NullPointerException: null
[2018-02-27 08:06:20,232] [WARN] [i.n.u.i.l.Slf4JLogger] [Null|127.0.0.1|Null|Null] - An exception was thrown by org.redisson.command.CommandAsyncService$15.operationComplete()
java.lang.NullPointerException: null
[2018-02-27 08:06:20,308] [WARN] [o.r.c.SentinelConnectionManager] [Null|127.0.0.1|Null|Null] - Skipped slave up 57.190.12.71:6399 for master redis://?:0 differs from current redis://57.190.12.72:6399

[2018-02-27 08:06:14,360] [ERROR] [i.u.s.a.LoggingExceptionHandler] [651d4b3e-6b90-4ab5-909e-46120aa1bcf8|10.4.0.2|Null|UAE10] - UT005023: Exception handling request to /connect
java.lang.IllegalArgumentException: session cannot be null
        at org.springframework.session.web.http.HttpSessionAdapter.<init>(HttpSessionAdapter.java:51)
        at org.springframework.session.web.http.SessionRepositoryFilter$SessionRepositoryRequestWrapper$HttpSessionWrapper.<init>(SessionRepositoryFilter.java:372)
        at org.springframework.session.web.http.SessionRepositoryFilter$SessionRepositoryRequestWrapper.getSession(SessionRepositoryFilter.java:308)
        at org.springframework.session.web.http.SessionRepositoryFilter$SessionRepositoryRequestWrapper.getSession(SessionRepositoryFilter.java:196)
        at javax.servlet.http.HttpServletRequestWrapper.getSession(HttpServletRequestWrapper.java:270)
        at org.springframework.security.web.context.HttpSessionSecurityContextRepository.loadContext(HttpSessionSecurityContextRepository.java:110)
        at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:100)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
        at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:56)
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
        at org.springframework.security.web.access.channel.ChannelProcessingFilter.doFilter(ChannelProcessingFilter.java:157)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
        at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:214)
        at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:177)
        at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:347)
        at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:263)
        at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
        at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
        at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99)
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
        at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
        at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
        at org.springframework.web.filter.HttpPutFormContentFilter.doFilterInternal(HttpPutFormContentFilter.java:108)
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
        at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
        at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
        at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:81)
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
        at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
        at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
        at org.springframework.session.web.http.SessionRepositoryFilter.doFilterInternal(SessionRepositoryFilter.java:146)
        at org.springframework.session.web.http.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:81)
        at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
        at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
        at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:197)
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
        at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
        at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
        at org.springframework.boot.actuate.autoconfigure.MetricsFilter.doFilterInternal(MetricsFilter.java:106)
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
        at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
        at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
        at io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:84)
        at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)
        at io.undertow.servlet.handlers.ServletChain$1.handleRequest(ServletChain.java:64)
        at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)
        at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131)
        at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)
        at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
        at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)
        at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64)
        at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60)
        at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77)
        at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43)
        at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
        at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
        at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292)
        at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81)
at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138)
        at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135)
        at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48)
        at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43)
        at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272)
        at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81)
        at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104)
        at io.undertow.server.Connectors.executeRootHandler(Connectors.java:332)
        at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:830)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:748)```
  

[2018-02-27 08:06:57,834] [ERROR] [i.u.s.a.LoggingExceptionHandler] [24b62c4f-f833-4463-bdab-ada972beeb49|10.4.0.2|Null|UAE9] - UT005023: Exception handling request to /connect/login
org.redisson.client.RedisTimeoutException: Unable to send command: (HGETALL) with params: [redisson_spring_session:cb50538c-5244-4cb2-953d-77b733fe5814] after 3 retry attempts
        at org.redisson.command.CommandAsyncService$8.run(CommandAsyncService.java:562)
        at io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:663)
        at io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:738)
        at io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:466)
        at java.lang.Thread.run(Thread.java:748)

[2018-02-27 08:07:02,365] [ERROR] [o.r.c.SentinelConnectionManager$8] [Null|127.0.0.1|Null|Null] - Can't add slave: redis://57.190.12.71:6399
org.redisson.client.RedisConnectionException: Unable to init enough connections amount! Only 31 from 32 were initialized. Redis server: /57.190.12.71:6399
        at org.redisson.connection.pool.ConnectionPool$2$1.operationComplete(ConnectionPool.java:138)
        at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:512)
        at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:486)
        at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:425)
        at io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:122)
        at org.redisson.misc.RedissonPromise.tryFailure(RedissonPromise.java:108)
        at org.redisson.connection.pool.ConnectionPool.promiseFailure(ConnectionPool.java:307)
        at org.redisson.connection.pool.ConnectionPool.access$400(ConnectionPool.java:54)
        at org.redisson.connection.pool.ConnectionPool$4.operationComplete(ConnectionPool.java:277)
        at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:512)
        at io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:505)
        at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:484)
        at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:425)
        at io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:122)
        at org.redisson.misc.RedissonPromise.tryFailure(RedissonPromise.java:108)
        at org.redisson.client.RedisClient$2$1$2.run(RedisClient.java:225)
        at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
        at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:463)
        at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:886)
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.lang.Thread.run(Thread.java:748)
Caused by: io.netty.channel.ConnectTimeoutException: connection timed out: /57.190.12.71:6399
        at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe$1.run(AbstractNioChannel.java:267)
        at io.netty.util.concurrent.PromiseTask$RunnableAdapter.call(PromiseTask.java:38)
        at io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:125)
        ... 6 common frames omitted

Issue Analytics

  • State:closed
  • Created 6 years ago
  • Comments:10 (5 by maintainers)

github_iconTop GitHub Comments

1reaction
SympathyForTheDevcommented, Mar 28, 2018

Hello @mrniko I confirm it’s fixed with 3.6.4, thanks

0reactions
mrnikocommented, Mar 21, 2018

I fixed https://github.com/redisson/redisson/issues/1272. Please check 3.0.0 branch

Read more comments on GitHub >

github_iconTop Results From Across the Web

Redisson Client with Sentinel throws exception - Stack Overflow
I wanted to know if I'm missing any thing in the redisson config or my sentinel servers are not configured well. java ·...
Read more >
https://raw.githubusercontent.com/redisson/redisso...
countExists()` method throws errors in cluster mode Fixed - Spring Data Redis ... `Can't find host in slaves!` error after failover with a...
Read more >
Java Client Can not Connect To Redis Sentinel Which Is ...
A single Sentinel not working well or not well connected can't failover a master without authorization from other Sentinels.Clients can connect to any...
Read more >
Release 0.1 moco_beta
Redis is a TCP server using the client-server model and what is called a Request/Response protocol. This means that usually a request is ......
Read more >
CHANGELOG.md · redisson-2.11.4 · mirrors / redisson / redisson ...
Fixed - Unable to send command error under heavy load using Redisson PRO ... onSubscribe should be invoked after failover process
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found
Previous page

What is difference between Rmap and RLocalCachedMap

Next page

Redisson client keeps running after shutdown invoked