question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Pulsar Java admin client should work with URL that points to a load balancer

See original GitHub issue

Expected behavior

When creating an instance of the Java Pulsar administrative client like this:

admin = PulsarAdmin.builder().serviceHttpUrl(pulsarAdminUrl).build();

it should be possible to supply a URL that references a load balancer e.g.

http://emodb-pulsar-databus-pulsar-cluster-1:8080

Actual behavior

If you do this however the admin client cannot communicate correctly with Pulsar and you will get unexepected end-of-file errors when trying to use the admin client to create topics etc. For example, in some of my code I try to create partitioned topics like this:

                try {admin.topics().createPartitionedTopic(FULL_MASTER_QUEUE_TOPIC_NAME, NUM_PARTITIONS); } catch (Throwable t) { _log.error("DefaultDatabus.constructor: caught exception " + t.getClass().getName() + " creating " + FULL_MASTER_QUEUE_TOPIC_NAME + ", message = " + t.getMessage());}
                try {admin.topics().createPartitionedTopic(FULL_RESOLVER_QUEUE_TOPIC_NAME, NUM_PARTITIONS); } catch (Throwable t) { _log.error("DefaultDatabus.constructor: caught exception " + t.getClass().getName() + " creating " + FULL_RESOLVER_QUEUE_TOPIC_NAME + ", message = " + t.getMessage());}
                try {admin.topics().createPartitionedTopic(FULL_RESOLVER_RETRY_QUEUE_TOPIC_NAME, NUM_PARTITIONS); } catch (Throwable t) { _log.error("DefaultDatabus.constructor: caught exception " + t.getClass().getName() + " creating " + FULL_RESOLVER_RETRY_QUEUE_TOPIC_NAME + ", message = " + t.getMessage());}

and this results in these errors appearing in the logs:

WARN  [2018-12-19 16:21:41,643] org.apache.pulsar.client.admin.internal.BaseResource: [http://emodb-pulsar-databus-pulsar-cluster-1:8080/admin/v2/persistent/public/default/master-queue/partitions] Failed to perform http put request: java.net.SocketException: Unexpected end of file from server
ERROR [2018-12-19 16:21:41,643] com.bazaarvoice.emodb.databus.core.DefaultDatabus: DefaultDatabus.constructor: caught exception org.apache.pulsar.client.admin.PulsarAdminException creating public/default/master-queue, message = java.net.SocketException: Unexpected end of file from server
WARN  [2018-12-19 16:21:41,649] org.apache.pulsar.client.admin.internal.BaseResource: [http://emodb-pulsar-databus-pulsar-cluster-1:8080/admin/v2/persistent/public/default/resolver-queue/partitions] Failed to perform http put request: java.net.SocketException: Unexpected end of file from server
ERROR [2018-12-19 16:21:41,649] com.bazaarvoice.emodb.databus.core.DefaultDatabus: DefaultDatabus.constructor: caught exception org.apache.pulsar.client.admin.PulsarAdminException creating public/default/resolver-queue, message = java.net.SocketException: Unexpected end of file from server
WARN  [2018-12-19 16:21:41,654] org.apache.pulsar.client.admin.internal.BaseResource: [http://emodb-pulsar-databus-pulsar-cluster-1:8080/admin/v2/persistent/public/default/resolver-retry-queue/partitions] Failed to perform http put request: java.net.SocketException: Unexpected end of file from server
ERROR [2018-12-19 16:21:41,654] com.bazaarvoice.emodb.databus.core.DefaultDatabus: DefaultDatabus.constructor: caught exception org.apache.pulsar.client.admin.PulsarAdminException creating public/default/resolver-retry-queue, message = java.net.SocketException: Unexpected end of file from server

Steps to reproduce

  1. Set up a Pulsar cluster in AWS
  2. Create an ELB for your Pulsar cluster that listens on ports 6650 and 8080
  3. Create a Route53 record set that provides an alias that points to your ELB
  4. In some Java code use the URL you set up in Route53 to create an admin client
  5. Attempt to perform an admin function like creating a topic using your client, you should see an end-of-file exception propagating up from the call
  6. Replace the URL that points to the ELB with a URL containing the IP address of one of the Pulsar cluster’s broker nodes and use that to create the admin client
  7. Now try to create the topic, it should work without errors

System configuration

Pulsar version: 2.2.0

Issue Analytics

  • State:closed
  • Created 5 years ago
  • Comments:12 (5 by maintainers)

github_iconTop GitHub Comments

1reaction
lxkakacommented, Jul 22, 2019

@sijie the problem was resolved.

0reactions
sijiecommented, Jul 21, 2019

@lxkaka is the problem addressed at your side?

Read more comments on GitHub >

github_iconTop Results From Across the Web

Pulsar configuration
Name Description Default exposePublisherStats Whether to enable topic level metrics. true statsUpdateFrequencyInSecs 60 statsUpdateInitialDelayInSecs 60
Read more >
Spring for Apache Pulsar
On a very high level, Spring for Apache Pulsar provides a PulsarTemplate for publishing to a Pulsar topic and a PulsarListener annotation for...
Read more >
pulsar broker always reconnect with pulsar-client-admin-original
My client use org.apache.pulsar.pulsar-client-admin-original:2.8.0 to get pulsar broker's info, and it works. My pulsar broker's version is ...
Read more >
Pulsar Advantages Over Kafka - ITNEXT
Working with offsets could be complicated if you just need a messaging system. Cluster re-balancing can impact the performance of connected ...
Read more >
Apache Pulsar - DigitalOcean Documentation
A simple client API with bindings for Java, Go, Python and C++. ... If you aren't logged in, this link will prompt you...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found