Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Consumer: Fails to subscribe after run()

See original GitHub issue

Hi guys, I have a follow up question to https://github.com/tulios/kafkajs/issues/336#issuecomment-490830101

I started to play around with kafkajs and tried to integrate it to our high-level solution. We need to dynamically subscribe to topics during runtime. To me it looks like subscribing to topics after consumer.run(…) was called does not add the new topic to the list of topics that get fetched in the ConsumerGroup:

kafkajs_subscribe_bug_1 (‘topic-test’ and ‘service__alpha’ gets subscribed before calling run(), ‘XXX’ after run())

Is it me doing it wrong, is the consumer not intended to be used in this way or did I stumble over a bug?

Thanks a lot in advance!

Issue Analytics

State:
Created 4 years ago
Comments:6 (2 by maintainers)

Top GitHub Comments

11reactions

Arkh1commented, Feb 13, 2020

@tulios - Could you elaborate on what you mean by restarting the consumer? Like the OP I too have a need for dynamically creating and subscribing to topics. I tried calling consumer.stop() followed by consumer.run(), but that didn’t seem to help. I also tried just creating a new instance of the same consumer and subscribing to all existing topics again. That also didn’t seem to work.

Also, has there been any updates on supporting dynamic topics in a more natural way?

0reactions

shaolinjrcommented, Nov 27, 2022

Anyone got solution for this??

Top Results From Across the Web

Kafka consumer api failed to subscribe to topic - Stack Overflow

I faced the same issue. I was using the kafka_2.12 jar version, when I downgrade it to kafka_2.11 it worked.

Consuming Messages - KafkaJS

This method has to be called after the consumer is initialized and is running (after consumer#run). await consumer.connect() await consumer.subscribe({ topics: ...

Kafka Consumer | Confluent Documentation

An Apache Kafka consumer group is a set of consumers which cooperate to consume data from some topics.

Introduction - Apache Kafka

Other servers run Kafka Connect to continuously import and export data as ... and consumers are those that subscribe to (read and process)...

KeyError while running snuba-subscription-consumer-events

Hi Sentry Team. We were missing running two sentry and snuba components (subscription-consumer-events and ...