question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Provide a setting in consumer configuration to specify where to start consuming messages

See original GitHub issue

Consumer provides a seek api for applications to seek to earliest or latest to consume messages. It would be good to add a setting in consumer configuration to simply application logic. It is useful for application development and any other similar required use cases.

Issue Analytics

  • State:closed
  • Created 6 years ago
  • Comments:7 (4 by maintainers)

github_iconTop GitHub Comments

2reactions
xiaozongyangcommented, Jan 25, 2018

@sijie Let me try to explain the intention first. We will add a new configuration item which is used by a consumer when subscribing a topic. In detail, we want our consumer program to determine from which place (earlies-the right beginning of message stream or latest-current consuming position) it should begin to consume rather than call the method seek explicitly. Is it the right thing you want? Expecting your reply 😃

1reaction
merlimatcommented, Jan 25, 2018

this task is more about adding a configuration setting, so when a consumer start, it would automatically start from earliest, latest or the cursor that current subscription left.

I would expose it in a slightly different angle, such as : “If the subscription does not exist, where do we want to initialize it (either latest or earliest)”

My rationale is that this setting goes with the code, so it’s not going to be practical to change it in different restarts of the same consumer.

For that I would suggest to add an option in ConsumerConfiguration named something like:

// Defaults to MessageId.latest
// only accepts MessageId.earliest
void setInitializeSubscriptionOn(MessageId msgId); 

or an alternative might be just a flag, I don’t know what’s better:

// defaults true, if passed false, means "earliest"
void setIinitializeSubscriptionOnLatest(boolean latest); 

With this, one can set the default behavior in the code and then forget about it.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Kafka Consumer Configurations for Confluent Platform
This topic provides the configuration parameters that are available for Confluent Platform. The Apache Kafka® consumer configuration parameters are ...
Read more >
Chapter 4. Kafka Consumers: Reading Data from Kafka
When we add a new consumer to the group, it starts consuming messages from partitions previously consumed by another consumer. The same thing...
Read more >
Working With the Kafka Console Consumer Platform For ...
On running the Kafka console consumer utility, the Kafka environment starts up a terminal window or command line prompt.
Read more >
Spring Boot and Kafka – Practical Example
Kafka Consumer configuration · [Omitted] Set up the Consumer properties in a similar way as we did for the Producer. · Create the...
Read more >
Start consuming only latest messages from Kafka Topic by ...
config.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "latest");. and then change you group.id and redeploy your consumer. it will start ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found