Manual commit - acknowledge() returns void and CommitFailedException happens after acknowledge() has already been called

Hello,

I am currently trying to write some code to prevent duplicate processing on the consumer side as mentioned in #858.

On the consumer side, there are no exactly once guarantees; you are on your own to implement idempotency; but this is relatively easy, just store the topic/partition/offset along with the data and check that you haven’t already processed this record. The topic/partition/offset are available in the Message<?> headers.

So I am implementing using manual commit. As per spring cloud stream kafka binder documentation, I am doing something very similar:

 @StreamListener(Sink.INPUT)
 public void process(Message<?> message) {
     Acknowledgment acknowledgment = message.getHeaders().get(KafkaHeaders.ACKNOWLEDGMENT, Acknowledgment.class);
     if (acknowledgment != null) {
         System.out.println("Acknowledgment provided");
         acknowledgment.acknowledge();
     }
 }

When I am debugging (on IDE) through my @StreamListener and manual acking, I sometimes see CommitFailedException due to rebalancing some time after acknowledgement.acknowledge() has already been stepped through. I know that CommitFailedException is related to configuration such as max.poll.interval.ms and others. I am going to set proper configs. But at the same time, I want to be able to handle this in code as well. And I am not using batch-mode, so I am processing 1 record at a time.

Knowing that acknowledge() method returns void, Is it possible to catch Exceptions that happen during/after acknowledge() ?

I think it would be nice and easy to do something like

try {
   ack.acknowledge();
}
catch (Exception e) {
    ... handle exception ...
}

[jskim1991]

@garyrussell Using AbstractMessageListenerContainer allows me to catch exceptions after calling acknowledge(). Once again, thank you so much.

[jskim1991]

@garyrussell here is my minimum set of a sample I have been testing all day just for reference. https://github.com/jskim1991/kafka-test-manual-commit/tree/master