Pulsar Java admin client should work with URL that points to a load balancer
See original GitHub issueExpected behavior
When creating an instance of the Java Pulsar administrative client like this:
admin = PulsarAdmin.builder().serviceHttpUrl(pulsarAdminUrl).build();
it should be possible to supply a URL that references a load balancer e.g.
http://emodb-pulsar-databus-pulsar-cluster-1:8080
Actual behavior
If you do this however the admin client cannot communicate correctly with Pulsar and you will get unexepected end-of-file errors when trying to use the admin client to create topics etc. For example, in some of my code I try to create partitioned topics like this:
try {admin.topics().createPartitionedTopic(FULL_MASTER_QUEUE_TOPIC_NAME, NUM_PARTITIONS); } catch (Throwable t) { _log.error("DefaultDatabus.constructor: caught exception " + t.getClass().getName() + " creating " + FULL_MASTER_QUEUE_TOPIC_NAME + ", message = " + t.getMessage());}
try {admin.topics().createPartitionedTopic(FULL_RESOLVER_QUEUE_TOPIC_NAME, NUM_PARTITIONS); } catch (Throwable t) { _log.error("DefaultDatabus.constructor: caught exception " + t.getClass().getName() + " creating " + FULL_RESOLVER_QUEUE_TOPIC_NAME + ", message = " + t.getMessage());}
try {admin.topics().createPartitionedTopic(FULL_RESOLVER_RETRY_QUEUE_TOPIC_NAME, NUM_PARTITIONS); } catch (Throwable t) { _log.error("DefaultDatabus.constructor: caught exception " + t.getClass().getName() + " creating " + FULL_RESOLVER_RETRY_QUEUE_TOPIC_NAME + ", message = " + t.getMessage());}
and this results in these errors appearing in the logs:
WARN [2018-12-19 16:21:41,643] org.apache.pulsar.client.admin.internal.BaseResource: [http://emodb-pulsar-databus-pulsar-cluster-1:8080/admin/v2/persistent/public/default/master-queue/partitions] Failed to perform http put request: java.net.SocketException: Unexpected end of file from server
ERROR [2018-12-19 16:21:41,643] com.bazaarvoice.emodb.databus.core.DefaultDatabus: DefaultDatabus.constructor: caught exception org.apache.pulsar.client.admin.PulsarAdminException creating public/default/master-queue, message = java.net.SocketException: Unexpected end of file from server
WARN [2018-12-19 16:21:41,649] org.apache.pulsar.client.admin.internal.BaseResource: [http://emodb-pulsar-databus-pulsar-cluster-1:8080/admin/v2/persistent/public/default/resolver-queue/partitions] Failed to perform http put request: java.net.SocketException: Unexpected end of file from server
ERROR [2018-12-19 16:21:41,649] com.bazaarvoice.emodb.databus.core.DefaultDatabus: DefaultDatabus.constructor: caught exception org.apache.pulsar.client.admin.PulsarAdminException creating public/default/resolver-queue, message = java.net.SocketException: Unexpected end of file from server
WARN [2018-12-19 16:21:41,654] org.apache.pulsar.client.admin.internal.BaseResource: [http://emodb-pulsar-databus-pulsar-cluster-1:8080/admin/v2/persistent/public/default/resolver-retry-queue/partitions] Failed to perform http put request: java.net.SocketException: Unexpected end of file from server
ERROR [2018-12-19 16:21:41,654] com.bazaarvoice.emodb.databus.core.DefaultDatabus: DefaultDatabus.constructor: caught exception org.apache.pulsar.client.admin.PulsarAdminException creating public/default/resolver-retry-queue, message = java.net.SocketException: Unexpected end of file from server
Steps to reproduce
- Set up a Pulsar cluster in AWS
- Create an ELB for your Pulsar cluster that listens on ports 6650 and 8080
- Create a Route53 record set that provides an alias that points to your ELB
- In some Java code use the URL you set up in Route53 to create an admin client
- Attempt to perform an admin function like creating a topic using your client, you should see an end-of-file exception propagating up from the call
- Replace the URL that points to the ELB with a URL containing the IP address of one of the Pulsar cluster’s broker nodes and use that to create the admin client
- Now try to create the topic, it should work without errors
System configuration
Pulsar version: 2.2.0
Issue Analytics
- State:
- Created 5 years ago
- Comments:12 (5 by maintainers)
Top Results From Across the Web
Pulsar configuration
Name Description Default
exposePublisherStats Whether to enable topic level metrics. true
statsUpdateFrequencyInSecs 60
statsUpdateInitialDelayInSecs 60
Read more >Spring for Apache Pulsar
On a very high level, Spring for Apache Pulsar provides a PulsarTemplate for publishing to a Pulsar topic and a PulsarListener annotation for...
Read more >pulsar broker always reconnect with pulsar-client-admin-original
My client use org.apache.pulsar.pulsar-client-admin-original:2.8.0 to get pulsar broker's info, and it works. My pulsar broker's version is ...
Read more >Pulsar Advantages Over Kafka - ITNEXT
Working with offsets could be complicated if you just need a messaging system. Cluster re-balancing can impact the performance of connected ...
Read more >Apache Pulsar - DigitalOcean Documentation
A simple client API with bindings for Java, Go, Python and C++. ... If you aren't logged in, this link will prompt you...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
@sijie the problem was resolved.
@lxkaka is the problem addressed at your side?