Support for closing a consumer without auto-committing offsets
See original GitHub issueWhen I destroy a consumer I want to be able to call close()
on it so it is removed from the consumer group (without having to wait for the session timeout).
Calling close triggers a offset commit when using auto-commit, which assumes you have finished processing all messages you have retrieved via poll()
or the iterator. However, if you are closing the consumer due to an error or interrupt you may not have finished processing the last message(s).
There doesn’t seem to be a way to close the consumer without committing offsets when using auto-commit.
Issue Analytics
- State:
- Created 7 years ago
- Comments:7 (5 by maintainers)
Top Results From Across the Web
How does kafka consumer auto commit work? - Stack Overflow
The auto-commit check is called in every poll and it checks that the time elapsed is greater than the configured time. If so,...
Read more >Kafka Consumers | Confluent Platform 3.1.1
The older Scala consumers are still supported for now, but they are not ... Sending periodic offset commits (if autocommit is enabled).
Read more >Chapter 4. Kafka Consumers: Reading Data from Kafka
During those seconds, no messages will be processed from the partitions owned by the dead consumer. When closing a consumer cleanly, the consumer...
Read more >Consuming Messages - KafkaJS
When suppling a regular expression, the consumer will not match topics created after the ... Promise<void> is used to commit offsets based on...
Read more >IT33662: ENABLE.AUTO.COMMIT AND ASYNC OFFSET ...
COMMIT AND ASYNC OFFSET COMMIT OPTIONS FOR KAFKA CONSUMER NODE. APAR status. Closed as ...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
I put up a small PR to address this. I agree that it diverges from the java client implementation and that autocommit-on-close is generally the intended use case. But hey we’re pythonistas, and adding an optional kwarg doesn’t seem too bad. Thoughts?
Fixed in #1031