Avoiding a rebalance if you need to wait longer than the heartbeat

Hi Folks, I have a problem that doesn’t happen with spring-kafka and I would like to know if there is a right way to do with spring-cloud-stream-kafka.

The scenario: I have business logic to wait for some specific time duration to process the next message. The problem: Using the approach bellow, if the time is greater than 10sec, the kafka start rebalance given the heartbeat process.

    @Override
    public void run(ApplicationArguments args) {
        new Thread(() -> doWork(errors, events)).start();
    }

    private void doWork(PollableMessageSource source, MessageChannel output) {
        while(true) {
            try {
                source.poll(message -> errorProcessor.process(message, output),
                        new ParameterizedTypeReference<Event>() {});
                
                Thread.sleep(sleepTime);  ///// If this time is greater than 10 sec, the
            } catch (InterruptedException e) {
                log.error("Fail to sleep thread: {}", e.getMessage());
            }
        }
    }

My founds: Looks like given I’m waiting with Thread.sleep() the main thread is locked and this is why the heartbeat time out and then Kafka restart to rebalance. Make senses?

With this, I have tested the spring-kafka directly and with it works fine the code below:

    @KafkaListener(topics = "topic", groupId = "group1")
    public void consume(String message) {
        logger.info(String.format("#### -> Consumed message-> %s", message));
        try {
            Thread.sleep(60*1000);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }

        logger.info("#### -> end waiting for...");
    }

I got this idea from this Github issue from spring-kafka and on the same page I could see the there was an implementation to do the heartbeat to Kafka in the background, so this is why I imagine works for that approach.

Do you know if there is somehow to do this same idea using spring-cloud-stream? or if something that is needed to implement it?

[sobychacko]

Closing this due to no recent activity.

[garyrussell]

Even stranger; I can’t reproduce it at all now, with or without logging 😖