question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Support for closing a consumer without auto-committing offsets

See original GitHub issue

When I destroy a consumer I want to be able to call close() on it so it is removed from the consumer group (without having to wait for the session timeout).

Calling close triggers a offset commit when using auto-commit, which assumes you have finished processing all messages you have retrieved via poll() or the iterator. However, if you are closing the consumer due to an error or interrupt you may not have finished processing the last message(s).

There doesn’t seem to be a way to close the consumer without committing offsets when using auto-commit.

Issue Analytics

  • State:closed
  • Created 7 years ago
  • Comments:7 (5 by maintainers)

github_iconTop GitHub Comments

1reaction
dpkpcommented, Mar 13, 2017

I put up a small PR to address this. I agree that it diverges from the java client implementation and that autocommit-on-close is generally the intended use case. But hey we’re pythonistas, and adding an optional kwarg doesn’t seem too bad. Thoughts?

0reactions
dpkpcommented, Mar 14, 2017

Fixed in #1031

Read more comments on GitHub >

github_iconTop Results From Across the Web

How does kafka consumer auto commit work? - Stack Overflow
The auto-commit check is called in every poll and it checks that the time elapsed is greater than the configured time. If so,...
Read more >
Kafka Consumers | Confluent Platform 3.1.1
The older Scala consumers are still supported for now, but they are not ... Sending periodic offset commits (if autocommit is enabled).
Read more >
Chapter 4. Kafka Consumers: Reading Data from Kafka
During those seconds, no messages will be processed from the partitions owned by the dead consumer. When closing a consumer cleanly, the consumer...
Read more >
Consuming Messages - KafkaJS
When suppling a regular expression, the consumer will not match topics created after the ... Promise<void> is used to commit offsets based on...
Read more >
IT33662: ENABLE.AUTO.COMMIT AND ASYNC OFFSET ...
COMMIT AND ASYNC OFFSET COMMIT OPTIONS FOR KAFKA CONSUMER NODE. APAR status. Closed as ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found