Connection to node -1 could not be established. Broker may not be available.
Explanation of the problem
This problem report describes an issue that occurs intermittently after a crash of an application while using sbt runAll to restart. The error message that appears on the logging console is “org.apache.kafka.clients.NetworkClient – Connection to node -1 could not be established. Broker may not be available.” The issue first occurred with the online-auction-scala example, and it now occurs frequently in new code based on it, with no easy way to reproduce it. The closest way to reproduce the issue seems to be to run the example, stop it with Ctrl-C to emulate an application crash, and then restart the application.
During startup using runAll, Kafka crashes, as seen in the logs. The error message in the server.log file suggests that a broker is already registered on the path /brokers/ids/0, indicating that either a broker ID that is already in use has been configured, or the broker has been shut down and restarted faster than the zookeeper timeout, causing it to appear as if it is re-registering. To solve the problem, deleting the target/lagom-dynamic-projects/lagom-internal-meta-project-kafka/target/zookeeper_data directory appears to be effective in cleaning the embedded Kafka data and restarting without the error.
The user is using Lagom 1.4.1, Java 1.8.0.144, and macOS High Sierra 10.13.3. If the error is related to a corruption of Zookeeper data, the user wants to know if there is a way to prevent this corruption or automatically repair the embedded Kafka while using the runAll sbt command.
Troubleshooting with the Lightrun Developer Observability Platform
Getting a sense of what’s actually happening inside a live application is a frustrating experience, one that relies mostly on querying and observing whatever logs were written during development.
Lightrun is a Developer Observability Platform, allowing developers to add telemetry to live applications in real-time, on-demand, and right from the IDE.
- Instantly add logs to, set metrics in, and take snapshots of live applications
- Insights delivered straight to your IDE or CLI
- Works where you do: dev, QA, staging, CI/CD, and production
Start for free today
Problem solution for Connection to node -1 could not be established. Broker may not be available.
When encountering the error message “Connection to node -1 could not be established. Broker may not be available,” it’s typically an indication that Kafka was unable to start up for various reasons. It is recommended to review the Kafka logs to determine the exact cause of the issue.
One common reason for this error is when the listeners or advertised.listeners on the config/server.properties file are not properly configured or used. In this case, it is important to check that the configuration parameters are set correctly. Additional information and details on how to properly configure these parameters can be found through the provided link.
In summary, if you are encountering the error message “Connection to node -1 could not be established. Broker may not be available” it is advisable to review the Kafka logs to determine the root cause of the issue. Additionally, it is important to ensure that the listeners and advertised.listeners parameters in the server.properties file are properly configured in order to avoid this error in the future.
Other popular problems with Lagom
Problem: Lagom project fails to start with a ClassNotFoundException
One common problem with Lagom is that the project fails to start with a ClassNotFoundException. This error usually happens because the Lagom project fails to load the required classes. The issue could be caused by an incorrect project configuration, such as the incorrect specification of dependencies or a missing dependency.
Solution:
To fix the issue, you should check the dependencies specified in your project’s build.sbt file and make sure that all required dependencies are included. Additionally, you can try to clean the project by running the “sbt clean” command and then rebuild it using the “sbt compile” command.
Problem: Lagom service is not reachable
Another common problem with Lagom is that the service is not reachable. This issue could be caused by a misconfiguration of the service endpoints or incorrect network settings.
Solution:
To resolve the problem, you should check the service endpoints and make sure they are correctly defined. You should also check your network settings to ensure that the service can be accessed from external networks. Additionally, you can try to restart the service and see if the issue persists.
Problem: Kafka-related issues
Lagom uses Kafka for communication between services, and as a result, some common problems may arise related to Kafka. One issue is the failure of Kafka to start up, which could happen for various reasons. To resolve this problem, you should review the Kafka logs for the reason for the failure. Another issue is related to incorrect Kafka configuration, such as incorrectly configured listeners or advertised.listeners on the config/server.properties file.
Solution:
To fix this issue, you should review the Kafka configuration and make sure that the listeners and advertised.listeners are correctly configured. You should also ensure that Kafka is running on the correct port and that the port is accessible from external networks.
A brief introduction to Lagom
Lagom is a modern microservices framework built on top of the popular Lightbend platform. It provides developers with a set of powerful tools to build highly scalable, resilient, and responsive microservices-based applications. Lagom is designed to address the challenges of developing distributed systems and to make it easier for developers to build microservices architectures.
One of the main features of Lagom is its support for reactive programming. It is built on top of the Akka toolkit and embraces the Reactive Streams specification. This means that Lagom applications are highly responsive, resilient, and elastic. They are able to handle large amounts of data and are able to scale up and down as needed to meet the demands of the application. Additionally, Lagom provides a set of libraries and tools for building event-driven, CQRS-based applications. It allows developers to build loosely coupled, highly scalable microservices that can communicate with each other through a message-driven architecture. Overall, Lagom is a powerful and flexible framework that enables developers to build highly responsive, scalable, and resilient microservices-based applications.
Most popular use cases for Lagom
Lagom can be used for building microservices-based applications. It provides a framework for building reactive microservices that are scalable, resilient, and responsive. With Lagom, developers can focus on writing business logic while the framework handles the underlying infrastructure such as service discovery, message brokering, and distributed data management. Lagom also provides built-in support for event sourcing and command query responsibility segregation (CQRS) patterns, allowing developers to implement scalable and highly available data stores.
Lagom can be used to create highly performant and scalable systems by leveraging reactive programming principles. It uses the reactive streams API to manage the flow of data between services, which helps to avoid backpressure and allows for efficient handling of large amounts of data. Lagom also supports asynchronous and non-blocking I/O operations, which ensures that services can handle a large number of concurrent requests without being blocked. Additionally, Lagom provides built-in support for Akka, an actor-based runtime that enables highly concurrent and distributed systems.
Lagom can be used for developing event-driven applications. It provides support for event sourcing and CQRS patterns, which are well suited for applications that need to handle a high volume of events. Lagom’s event sourcing framework allows developers to capture all changes to the application’s state as a sequence of events. These events can then be used to recreate the application’s state at any point in time, which makes it easy to implement auditing, debugging, and replay features. The CQRS pattern allows for the separation of the read and write models, which enables highly optimized and scalable read operations. The following code block demonstrates how to define an event in Lagom:
case class OrderCreated(orderId: String, customerId: String, items: Seq[OrderItem])
object OrderCreated {
implicit val format: Format[OrderCreated] = Json.format[OrderCreated]
}
It’s Really not that Complicated.
You can actually understand what’s going on inside your live applications.