Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Multiple topics consumer not able to receive the earliest message (python client)

See original GitHub issue

Describe the bug When using multiple topics with the Python client and initial_position=Earliest, new subscriptions still begin with the latest message. Related: #3494, #3712.

To Reproduce Code for consumer:

rx = re.compile('persistent://public/default/topic-.*')
sub = pulsar_client.subscribe(rx, 'sub', pulsar.ConsumerType.Shared,
        initial_position=pulsar.InitialPosition.Earliest)

When a new topic is created, it is auto-discovered, as expected. However, all messages sent from the creation of the topic, until the subscription is made, are lost.

Expected behavior When using initial_position=Earliest with a topic pattern, all new topic subscriptions should begin with the earliest message.

Issue Analytics

State:
Created 4 years ago
Comments:7 (5 by maintainers)

Top GitHub Comments

1reaction

dhagrowcommented, Jan 9, 2020

@jiazhai this is 2.4.2.

0reactions

tisonkuncommented, Dec 9, 2022

Closed as stale. Please create a new issue if it’s still relevant to the maintained versions.

Top Results From Across the Web

kafka-python consumer not receiving messages - Stack Overflow

I was testing it with a topic with more than one partition, it so happens that the issue arises only when the producer...

Kafka: Consume Messages From Multiple Queues - Medium

A. Yes, Kafka's design allows consumers from one consumer group to consume messages from multiple topics. The protocol underlying consumer.

Chapter 4. Kafka Consumers: Reading Data from Kafka

When multiple consumers are subscribed to a topic and belong to the same consumer group, each consumer in the group will receive messages...

KafkaConsumer — kafka-python 1.2.3 documentation

The java client defaults to 32768. consumer_timeout_ms (int) – number of millisecond to throw a timeout exception to the consumer if no message...

Messaging - Apache Pulsar

In this pattern, producers publish messages to topics; consumers subscribe to ... If the acknowledgment is not received, the producer treats the sending ......