question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

[Question] Multiple Kafka clusters with the same topic name and namespace

See original GitHub issue

I have two Kafka clusters, Kafka-a, and Kafka-b. They are in a namespace, and the auto.create.topics.enable the option is turned on, and both use a topic (user) with the same name. When the user topic is automatically set by Kafka After creation, the topics operator will create a corresponding KafkaTopic resource in the k8s namespace to manage the corresponding topic, and then they will have the same name

Is there a way to add a prefix to the KafkaTopic created by the topicOperator in the k8s namespace, similar to

kakfa-a-user kakfa-b-user

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:10 (5 by maintainers)

github_iconTop GitHub Comments

1reaction
AnyISalIncommented, Sep 16, 2020

From your description I’m not sure I understand why would each project require multiple Kafka clusters. I can see why it might be an inconvenience, but I do not think there is anything you can do about it right now.

You can also disable the topic operator by removing the topicOperator: {} section from the Kafka CR.

Just like openshift, we use the namespace of Kubernetes to distinguish projects. In most cases, there will not be multiple instances of Strimzi Kafka in a project.

0reactions
scholzjcommented, Sep 21, 2020

BTW: spec.topicName is optional - so in many resources it will not be there.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Is it better to split Kafka clusters? | Red Hat Developer
Therefore, multi-topic subscriptions or load balancing across the consumers in a consumer group are possible only within a single Kafka cluster.
Read more >
Managing Topics across Multiple Kafka Clusters | 6.3.x
You can distribute messages across multiple clusters. It can be handy to have a copy of one or more topics from other Kafka...
Read more >
Manage Schemas for Topics in Control Center
Following is a detailed example of a functional multi-cluster Schema Registry setup with two Kafka clusters connected to Control Center, one the controlcenter....
Read more >
Documentation - Apache Kafka
A two server Kafka cluster hosting four partitions (P0-P3) with two ... Let's create a topic named "test" with a single partition and...
Read more >
Deploying and Upgrading (0.32.0) - Strimzi
If you use Strimzi to deploy multiple Kafka clusters into the same namespace, enable the Topic Operator for only one Kafka cluster or...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found