Behavior of redelivery and ackTimeout in Java client

Expected behavior

With the Java client, when ackTimeout is specified for a consumer, unacked individual messages should be resent only after ackTimeout has expired.

Actual behavior

Unacked messages are all resent at once every ackTimeout period.

Steps to reproduce

Run this program

public class UnackedTrackerIssue {
    private static ExecutorService executor = Executors.newFixedThreadPool(10);

    public static void main(String[] args) throws PulsarClientException, InterruptedException {

        final String topic = "my-topic";

        PulsarClient client = PulsarClient.builder()
                .serviceUrl("pulsar://localhost:6650")
                .build();

        Consumer<byte[]> consumer = client.newConsumer()
                .topic(topic)
                .subscriptionName(UUID.randomUUID().toString())
                .ackTimeout(5, TimeUnit.SECONDS)
                .subscribe();

        receive(consumer);

        Producer<byte[]> producer = client.newProducer()
                .topic(topic)
                .create();

        for(int i=0;i<5; i++) {
            Thread.sleep(1000);
            producer.send(("" + i).getBytes());
        }

    }

    private static void receive(Consumer<byte[]> consumer) {
        consumer.receiveAsync()
                .thenAccept(msg -> {
                    System.out.println("" + Instant.now() + " consume " + new String(msg.getData()));
                    executor.execute(() -> receive(consumer));
                });
    }
}

Output is something like:

2018-11-26T16:51:04.555Z consume 0 2018-11-26T16:51:05.552Z consume 1 2018-11-26T16:51:06.558Z consume 2 2018-11-26T16:51:07.565Z consume 3 2018-11-26T16:51:08.323Z consume 0 2018-11-26T16:51:08.324Z consume 1 2018-11-26T16:51:08.325Z consume 2 2018-11-26T16:51:08.329Z consume 3 2018-11-26T16:51:08.574Z consume 4 2018-11-26T16:51:13.327Z consume 0 2018-11-26T16:51:13.329Z consume 1 2018-11-26T16:51:13.330Z consume 2 2018-11-26T16:51:13.330Z consume 3 2018-11-26T16:51:13.331Z consume 4

The delay of redelivery between the message “3” occurences should be about 5 seconds but the first time there’s only 1s.

Code that results in this behavior : https://github.com/apache/pulsar/blob/0ab2325fa33231f1c69782e081483012467dfcab/pulsar-client/src/main/java/org/apache/pulsar/client/impl/UnAckedMessageTracker.java#L87

System configuration

Pulsar version: master https://github.com/apache/pulsar/commit/fe2c8ee4d37e2a45dfb528592915746827416e18

[jiazhai]

Seems indeed as @cbornet mentioned, changes in PR #2590 make it worse. I would like to use this issue to revert changes in #2590.

But as @codelipenghui requirement, we should make a more accurate redeliver. A way todo this is that: split acktimeout into several time-slots, and introduce a timewheel to do the redeliver for each time-slots, the more time-slots user set, the more accurate it will be.

[jiazhai]

close this issue since penghui’s prs merged.