Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

KIP-62 / KAFKA-3888: Allow consumer to send heartbeats from a background thread

See original GitHub issue

Allows consumers to take their time processing messages without being timed out from their consumer group.

max_poll_records is a decent workaround for most of the pain here, but it’d still be nice to add this for consumers that have inconsistent message processing times.

Related issues: #872, #544

Issue Analytics

State:
Created 7 years ago
Reactions:4
Comments:27 (3 by maintainers)

Top GitHub Comments

3reactions

jeffwidmancommented, Sep 1, 2017

I would like to start working on this, we have multiple services at my day job that are impacted by this (and just realized there may be more due to #1039).

Unfortunately, I’ve never really worked with Java, so will be a bit slow trying to figure this out.

@tvoinarovskyi do you have any pointers on where to look in the Java code for where to start changing things compared to how they are in kafka-python today?

2reactions

tvoinarovskyicommented, Feb 8, 2017

Hey @jeffwidman Sorry, but GIL will not block you from doing IO in a separate thread. At least if you are not doing something crazy like passing all messages in a pure C code, that processes them. Python will yield GIL after some time (python3) or instruction count (python2), so as long as you are not stuck in C code it should be OK.