Pulsar standalone fails to start up when I set clusterName != "standalone"
See original GitHub issueDescribe the bug
When setting clusterName
in standalone mode, the server fails to start.
To Reproduce Steps to reproduce the behavior:
- Deploy in a kubernetes cluster.
- Set the environment variable:
clusterName=whatever
- Set pod command to:
sh -c '/pulsar/bin/apply-config-from-env.py /pulsar/conf/standalone.conf; exec /pulsar/bin/pulsar standalone;'
Tested today with images:apachepulsar/pulsar-standalone:latest
andapachepulsar/pulsar-all:latest
Expected behavior
The pod starts up with cluster-name: whatever
(something else than standalone
)
Screenshots From the log:
[/pulsar/conf/standalone.conf] Applying config clusterName = jgaa-west
[AppClassLoader@18b4aac2] info AspectJ Weaver Version 1.9.2 built on Wednesday Oct 24, 2018 at 15:43:33 GMT
...
11:04:16.797 [ForkJoinPool.commonPool-worker-1] WARN org.apache.pulsar.broker.web.PulsarWebResource - Namespace missing local cluster name in clusters list: local_cluster=jgaa-west ns=public/functions clusters=[standalone]
11:04:16.816 [pulsar-web-68-5] INFO org.eclipse.jetty.server.RequestLog - 127.0.0.1 - - [10/Mar/2021:11:04:16 +0000] "PUT /admin/v2/persistent/public/functions/assignments HTTP/1.1" 412 60 "-" "Pulsar-Java-v2.7.0" 79
11:04:16.825 [AsyncHttpClient-87-1] WARN org.apache.pulsar.client.admin.internal.BaseResource - [http://localhost:8080/admin/v2/persistent/public/functions/assignments] Failed to perform http put request: javax.ws.rs.ClientErrorException: HTTP 412 Precondition Failed
11:04:16.834 [main] ERROR org.apache.pulsar.functions.worker.WorkerService - Error Starting up in worker
org.apache.pulsar.client.admin.PulsarAdminException$PreconditionFailedException: Namespace does not have any clusters configured
at org.apache.pulsar.client.admin.internal.BaseResource.getApiException(BaseResource.java:225) ~[org.apache.pulsar-pulsar-client-admin-original-2.7.0.jar:2.7.0]
at org.apache.pulsar.client.admin.internal.BaseResource$1.failed(BaseResource.java:133) ~[org.apache.pulsar-pulsar-client-admin-original-2.7.0.jar:2.7.0]
Desktop (please complete the following information):
- Debian Buster / Kubernetes 1.19.0 / 1.20.0
Additional context It appears to me that the problem is in the initialization of the “extras” in the standalone deployment. For my use-case, it would be nice to have a switch to disable that. My aim is to develop and test a multi-region application that use Pulsar for replication between data centers. For development however, I want to use a single minikube cluster, and deploy the whole application in different k8s name-spaces - one for each region/“data center”. Using the standalone Pulsar option is simpler and faster than deploying a full Pulsar cluster in each namespace. Especially during development, when I may redeploy the app 20 times in a single day.
Issue Analytics
- State:
- Created 3 years ago
- Comments:9 (4 by maintainers)
Top GitHub Comments
I have created a ticket for it. Hopefully it will be fixed in not too long.
Hi, I am working on a distributed cloud app where Pulsar is one of many components. In production, Pulsar is deployed with Helm Charts. For my local development setup, I need to test and debug my components, and they use Pulsar. Therefore, I have been deploying Pulsar standalone, as this is pretty straight forward. It has worked fine for a long time - both with docker-compose and kubernetes.
Now, I am working on the code related to distributed processing, and I need to deploy (locally) several clusters. That means that the Pulsar cluster names cannot all be “standalone”. I prefer to use the standalone alternative for development, because it’s faster to deploy and simpler to use. As a work-around for now, I am deploying a normal Pulsar with zookeeper, bookkeeper and broker, each with 1 instance, using a custom built deployer.
But still, it would be nice if standalone Pulsar let me specify the cluster name. The option is documented, so when it don’t work, I assume it is a bug 😃