Broken subsequence of delayed messages

Describe the bug Delayed messages to shared topic breaks subsequence of messages and delivered earlier than expected.

To Reproduce Steps to reproduce the behavior:

1. Setup local standalone pulsar
2. Send messages every 20ms with the same deliverAt timestamp 10000ms and with payload of millisecond when sending was started.
3. Log received messages payload.

I start receiving all messages after 9 seconds and it I have in log:

9001ms
1ms
21ms
41ms
61ms
81ms
...
8981ms
(here should be 9001ms)
9021ms
9041ms
...

Expected behavior Two expectations:

1. Most important I expect first message to be received is message ‘1ms’ and message ‘9001ms’ after ‘8981ms’.
2. I expect start receiving messages after 10 seconds

Issue is reproduced in 2.8.0 and 2.9.1

[mattisonchao]

Hi, @zubchenok

I have already reproduced both problems, I will try to fix them ASAP.

[mattisonchao]

Hi, @zubchenok Sorry for missing this issue. The second issue:

I expect start receiving messages after 10 seconds

You can set isDelayedDeliveryDeliverAtTimeStrict=true to solve this problem.(broker version greater than or equals to 2.9.3)#16068 or set delayedDeliveryTickTimeMillis=1(broker version less than or equals to 2.9.3)