Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.
And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.
[feature] Multi topics and partitions commit
See original GitHub issueIn order to prevent messages lost. I’m reimplementing the offset commit logic on userland (I commit only once the messages are processed, not received).
I’m using the 'enable.auto.commit': false global option to disable the “auto commit”. I’m also batching the commit of the offsets evert X messages or Y seconds. In order to do so, I call the async consumer.commit() API multiple times (one time per (topic, partition) tuple).
I have noticed that the “auto commit” logic commit on different topics and partitions at the same time. I was looking into using such API instead of calling consumer.commit() X times. It would be more efficient.
Issue Analytics
- State:
- Created 6 years ago
- Comments:6
Top Results From Across the Web
Top Related Medium Post
Top Related StackOverflow Question
Troubleshoot Live Code
Top Related Reddit Thread
Top Related Hackernoon Post
Top Related Tweet
Top Related Dev.to Post
Top Related Hashnode Post
Support added with #351
Well, it isn’t a 1:1 request-per-call necessarily, so we can definitely get more performance out of commits.
https://github.com/edenhill/librdkafka/blob/master/src-cpp/rdkafkacpp.h#L1685
There is a method I can bind to that will take an array so I will go ahead implementing that. Shouldn’t be too long and it will make it in the next release!