Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Zipkin RabbitMQ Sender fails to reconnect with RabbitMQ broker

See original GitHub issue

Describe the bug RabbitMQ fails to reconnect with RabbitMQ broker on connection reset. RabbitConnectionFactoryBean in Spring AMQP, which is used in RabbitAutoConfiguration to configure ConnectionFactory for RabbitMQ, disables automatic connection recovery functionality provided by rabbit mq client library. Since, Spring AMQP has its own recovery mechanism, it disables connection recovery functionality provided by the client library. Issue was found in Spring Cloud Version: 2.x.

Dropped 18 spans due to AlreadyClosedException(connection is already closed due to connection error; cause: java.net.SocketException: Connection reset) com.rabbitmq.client.AlreadyClosedException: connection is already closed due to connection error; cause: java.net.SocketException: Connection reset? at 
com.rabbitmq.client.impl.AMQChannel.ensureIsOpen(AMQChannel.java:253) ~[amqp-client-5.4.3.jar!/:5.4.3]? at 
com.rabbitmq.client.impl.AMQChannel.transmit(AMQChannel.java:422) ~[amqp-client-5.4.3.jar!/:5.4.3]? at 
com.rabbitmq.client.impl.ChannelN.basicPublish(ChannelN.java:704) ~[amqp-client-5.4.3.jar!/:5.4.3]? at com.rabbitmq.client.impl.ChannelN.basicPublish(ChannelN.java:679) ~[amqp-client-5.4.3.jar!/:5.4.3]? at 
com.rabbitmq.client.impl.ChannelN.basicPublish(ChannelN.java:669) ~[amqp-client-5.4.3.jar!/:5.4.3]? at zipkin2.reporter.amqp.RabbitMQSender$RabbitMQCall.publish(RabbitMQSender.java:260) ~[zipkin-sender-amqp-client-2.7.3.jar!/:na]? at 
zipkin2.reporter.amqp.RabbitMQSender$RabbitMQCall.doExecute(RabbitMQSender.java:255) ~[zipkin-sender-amqp-client-2.7.3.jar!/:na]? at 
zipkin2.reporter.amqp.RabbitMQSender$RabbitMQCall.doExecute(RabbitMQSender.java:247) ~[zipkin-sender-amqp-client-2.7.3.jar!/:na]? at 
zipkin2.Call$Base.execute(Call.java:379) ~[zipkin-2.9.3.jar!/:na]? at 
zipkin2.reporter.AsyncReporter$BoundedAsyncReporter.flush(AsyncReporter.java:286) ~[zipkin-reporter-2.7.3.jar!/:na]? at 
zipkin2.reporter.AsyncReporter$Builder$1.run(AsyncReporter.java:190) [zipkin-reporter-2.7.3.jar!/:na]?

Issue Analytics

State:
Created 4 years ago
Reactions:1
Comments:7 (3 by maintainers)

Top GitHub Comments

1reaction

tomekstankowskicommented, Oct 14, 2022

I’ve experienced this issue in Spring Cloud 2021.0.4 and Spring Boot 2.7.4. I agree with @sufiannazim that it’s an issue of Spring Sleuth. Most developers will use RabbitMQ Zipkin sender in conjunction with Spring AMQP autoconfiguration and won’t be aware of the issue until it hits them. I think this should be documented at least.

0reactions

sufiannazimcommented, Feb 5, 2020

Yes, user can create a custom ConnectionFactory for Zipkin sender. But, Spring Cloud Sleuth default configuration for Zipkin should ensure that the connection factory used for RabbitMQ has a connection recovery support enabled.

Top Results From Across the Web

How to resolve RabbitMQ Server connection error in Spring ...

When I try to start the service, I'm getting " "AsyncReporter{RabbitMQSender{addresses=[localhost:5672], queue=zipkin}}. Unable to establish ...

Understanding Distributed Tracing | Microservices with Spring ...

Spring Cloud Sleuth can send trace information to Zipkin either synchronously over HTTP, or asynchronously using a message broker such as RabbitMQ or...

SWIS Issues - Failed to connect to Broker for the Orion Platform

1. Uninstall "RabbitMQ Server (SolarWinds Distribution)" from Programs and Features (take note of the version of RabbitMQ you are uninstalling ...

Tracing a Reactive Kotlin App with Spring Cloud Sleuth

This guide will discuss RSocket tracing with Spring Cloud Sleuth and Zipkin. We will ship data to Zipkin through HTTP/REST, Kafka, and RabbitMQ....

Spring Cloud Sleuth Reference Documentation

spring.zipkin.sender.type: kafka. If you want Sleuth over RabbitMQ, add the spring-cloud-starter-sleuth , spring-cloud-sleuth-zipkin and spring-rabbit ...