question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

move KafkaConnect and MirrorMaker 1.0/2.0 to statefulsets

See original GitHub issue

Statefulset allows having predictable POD names. That, in turn, allows using pod name as group.instance.id and enable static membership that was designed for cloud applications.

Issue Analytics

  • State:open
  • Created 2 years ago
  • Reactions:2
  • Comments:10 (8 by maintainers)

github_iconTop GitHub Comments

1reaction
scholzjcommented, Oct 12, 2022

I do not think this has anything to do with this issue. The coordinator simply changes while the Kafka broker restarts. If you have something more, please start a discussion or something and we can continue there: https://github.com/strimzi/strimzi-kafka-operator/discussions

0reactions
kinihuncommented, Oct 12, 2022

This is still an issue with StrimziPodSets, when a k8s cluster is upgraded, the nodes will change with rolling update hence the brokers will move around. There is no way to uniquely set the group.instance.id for each KafkaConnect pod forming the Connect deployment.

I’m not sure I follow what do you mean. Connect does not use StrimziPodSets. That is just future plan. If you think why that would nto help, please elaborate more on why because it is not obvious.

Apologies for the lack of clarity, the issue I am observing is quite similar and it has to do with KafkaConnect. As you can see from the error below, the Group coordinator is fixed hence the consumer cannot locate it after a cluster upgrade involving nodes undergoing rolling upgrade. I am using Strimzi 0.30.0 with Kafka version 3.2.0 hence why I mentioned StrimziPodSet which is now default. This issue is preventing my Connector tasks from executing at polling intervals.

"2022-10-11 09:28:28,738 INFO [Consumer clientId=consumer-prod-kafka-1, groupId=prod-kafka] Discovered group coordinator prod-kafka-kafka-1.prod-kafka-kafka-brokers.kafka.svc:9093 (id: 2147483646 rack: null) (org.apache.kafka.clients.consumer.internals.ConsumerCoordinator) [KafkaBasedLog Work Thread - prod-kafka-connect-offsets]" "2022-10-11 09:28:28,675 INFO [Consumer clientId=consumer-prod-kafka-2, groupId=prod-kafka] Discovered group coordinator prod-kafka-kafka-1.prod-kafka-kafka-brokers.kafka.svc:9093 (id: 2147483646 rack: null) (org.apache.kafka.clients.consumer.internals.ConsumerCoordinator) [KafkaBasedLog Work Thread - prod-kafka-connect-status]" "2022-10-11 09:28:28,637 INFO [Consumer clientId=consumer-prod-kafka-1, groupId=prod-kafka] Discovered group coordinator prod-kafka-kafka-1.prod-kafka-kafka-brokers.kafka.svc:9093 (id: 2147483646 rack: null) (org.apache.kafka.clients.consumer.internals.ConsumerCoordinator) [KafkaBasedLog Work Thread - prod-kafka-connect-offsets]" "2022-10-11 09:28:28,637 INFO [Consumer clientId=consumer-prod-kafka-1, groupId=prod-kafka] Group coordinator prod-kafka-kafka-1.prod-kafka-kafka-brokers.kafka.svc:9093 (id: 2147483646 rack: null) is unavailable or invalid due to cause: coordinator unavailable.isDisconnected: false. Rediscovery will be attempted. (org.apache.kafka.clients.consumer.internals.ConsumerCoordinator) [KafkaBasedLog Work Thread - prod-kafka-connect-offsets]" "2022-10-11 09:28:28,637 INFO [Consumer clientId=consumer-prod-kafka-1, groupId=prod-kafka] Requesting disconnect from last known coordinator prod-kafka-kafka-1.prod-kafka-kafka-brokers.kafka.svc:9093 (id: 2147483646 rack: null) (org.apache.kafka.clients.consumer.internals.ConsumerCoordinator) [KafkaBasedLog Work Thread - prod-kafka-connect-offsets]" "2022-10-11 09:28:28,636 INFO [Consumer clientId=consumer-prod-kafka-1, groupId=prod-kafka] Node 2147483646 disconnected. (org.apache.kafka.clients.NetworkClient) [KafkaBasedLog Work Thread - prod-kafka-connect-offsets]"

Read more comments on GitHub >

github_iconTop Results From Across the Web

Migrate from MirrorMaker to Replicator
This topic provides examples of how to migrate from an existing datacenter that is using MirrorMaker to Replicator.
Read more >
Migrating clusters using Apache Kafka's MirrorMaker
Makes use of the Apache Kafka Connect framework and ecosystem. Detects new topics and partitions. Automatically syncs topic configuration between clusters.
Read more >
Chapter 2. Deployment configuration Red Hat AMQ 2021.q2
Kafka Connect clusters with Source2Image support; Kafka MirrorMaker; Kafka Bridge; Cruise Control. Note. Labels applied to a custom resource are also applied to ......
Read more >
How to migrate consumer offsets using MirrorMaker 2.0?
With Kafka 2.7.0, I am using MirroMaker 2.0 as a Kafka-connect connector to replicate all the topics from the primary Kafka cluster to...
Read more >
MirrorMaker 2 on Kafka Connect - MSK Labs
Setup Kafka Connect cluster required for MirrorMaker2. Configure and start MirrorMaker2 connectors. (Optional) Start MM2 Consumer Group Offset Sync ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found