Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Consumer issues with `conf.rebalance_cb = true`

See original GitHub issue

if conf.rebalance_cb is set to true the consumer does not emit the rebalance event - which could be used to mimick seek by using assign as suggested by @edenhill

BTW with conf.rebalance_cb = true, the consumer calls an undefined unassign` JS function

kafka-consumer.js line 54

        self.unassign(e.assignment);

Issue Analytics

State:
Created 7 years ago
Comments:15

Top GitHub Comments

1reaction

pmidgecommented, Mar 10, 2017

Here you go. Not strictly a test case per se, but 100% representative of our use case.

In the directory containing the unpacked rebalance.js, run npm i node-rdkafka@0.8.0 underscore.
Open rebalance.js and modify topics and config to suit your environment.
node rebalance.js.

Out of the box, the repro is configured to use an internal rebalance handler. This should repro the failure I initially reported. You can modify rebalance behavior by referring to instructions at line 239.

ETA: Node version is 6.9.4

rebalance.js.gz

0reactions

edoardocomarcommented, May 18, 2017

I had used conf.rebalance_cb = true with the current 0.10.2 release and the event was emitted as expected. I’m closing this old issue

Top Results From Across the Web

confluent-kafka-go/consumer.go at master - GitHub

// AssignmentLost returns true if current partition assignment has been lost. // This method is only applicable for use with a subscribing consumer...

How to Implement Kafka Consumer with Partition Rebalance ...

When a rebalance occurs, the consumer might reprocess the same message again. This might not be a problem, if the downstream service is...

kafka - Go Documentation Server

Package kafka provides high-level Apache Kafka producer and consumers using ... rebalanceCb RebalanceCb) error: func (c *Consumer) SubscribeTopics(topics ...

edenhill/librdkafka - Gitter

After the topic creation, in < 12 seconds the rebalance cb was called and the ... consumed at start up for a set...

Kafka consumer, very long rebalances - Stack Overflow

I suspect your cluster version is at least 0.10.1.0 as I see max.poll.interval.ms in your consumer configuration which was introduced in ...