Reader: HasMessageAvailableAsync returns True but ReadNextAsync never returns when using StartMessageFromRollbackDuration
See original GitHub issueI am not sure if this is a Pulsar issue or issue with Pulsar.Client. Hoping someone can help point me in the right direction!
The below function works perfectly when using StartMessageId(MessageId.Earliest)
. If I try to use StartMessageFromRollbackDuration
like below, the HasMessageAvailableAsync()
will return true but then the ReadNextAsync
will block indefinitely, never returning a message. There is only one message in the queue at the time this runs, if that helps as well.
{
TimeSpan rollbackDuration = DateTime.Now - startReadtime;
await using IReader<byte[]> reader = await _readerBuilder
.StartMessageFromRollbackDuration(rollbackDuration)
//.StartMessageId(MessageId.Earliest)
.CreateAsync();
while (await reader.HasMessageAvailableAsync())
{
Message<byte[]> received = await reader.ReadNextAsync();
if (CheckMessage(received, messageCheck))
{
return true;
}
}
return false;
}
Here are the logs from my Standalone Pulsar image running in Docker from the point of creating the read to attempting to read the next:
20:05:18.932 [pulsar-io-51-2] INFO org.apache.pulsar.broker.service.ServerCnx - [/192.168.176.8:33082] Subscribing on topic persistent://public/default/block-schedule-activity / reader-9b5a98d18f 20:05:18.934 [pulsar-io-51-2] INFO org.apache.pulsar.broker.service.persistent.PersistentTopic - [persistent://public/default/block-schedule-activity][reader-9b5a98d18f] Creating non-durable subscription at msg id null 20:05:18.934 [pulsar-io-51-2] INFO org.apache.bookkeeper.mledger.impl.NonDurableCursorImpl - [public/default/persistent/block-schedule-activity] Created non-durable cursor read-position=10:1 mark-delete-position=10:0 20:05:18.934 [pulsar-io-51-2] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/block-schedule-activity] Opened new cursor: NonDurableCursorImpl{ledger=public/default/persistent/block-schedule-activity, ackPos=10:0, readPos=10:1} 20:05:18.973 [bookkeeper-ml-workers-OrderedExecutor-1-0] INFO org.apache.pulsar.broker.service.persistent.PersistentMessageFinder - [persistent://public/default/block-schedule-activity][reader-9b5a98d18f] Found position 10:0 closest to provided timestamp 1594238715935 20:05:18.973 [bookkeeper-ml-workers-OrderedExecutor-1-0] INFO org.apache.pulsar.broker.service.persistent.PersistentSubscription - [persistent://public/default/block-schedule-activity][reader-9b5a98d18f] Successfully disconnected consumers from subscription, proceeding with cursor reset 20:05:18.973 [bookkeeper-ml-workers-OrderedExecutor-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [public/default/persistent/block-schedule-activity] Initiate reset position to 10:1 on cursor reader-9b5a98d18f 20:05:18.975 [bookkeeper-ml-workers-OrderedExecutor-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [public/default/persistent/block-schedule-activity] reset position to 10:1 before current read position 10:1 on cursor reader-9b5a98d18f 20:05:18.976 [bookkeeper-ml-workers-OrderedExecutor-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [public/default/persistent/block-schedule-activity-reader-9b5a98d18f] Rewind from 10:1 to 10:1 20:05:18.976 [bookkeeper-ml-workers-OrderedExecutor-1-0] INFO org.apache.pulsar.broker.service.persistent.PersistentTopic - [persistent://public/default/block-schedule-activity] There are no replicated subscriptions on the topic 20:05:18.976 [bookkeeper-ml-workers-OrderedExecutor-1-0] INFO org.apache.pulsar.broker.service.persistent.PersistentTopic - [persistent://public/default/block-schedule-activity][reader-9b5a98d18f] Created new subscription for 1 20:05:18.976 [bookkeeper-ml-workers-OrderedExecutor-1-0] INFO org.apache.pulsar.broker.service.ServerCnx - [/192.168.176.8:33082] Created subscription on topic persistent://public/default/block-schedule-activity / reader-9b5a98d18f 20:05:30.334 [SyncThread-7-1] INFO org.apache.bookkeeper.bookie.EntryLogManagerBase - Creating a new entry log file because current active log channel has not initialized yet 20:05:30.339 [SyncThread-7-1] INFO org.apache.bookkeeper.bookie.EntryLoggerAllocator - Created new entry log file data/standalone/bookkeeper0/current/0.log for logId 0. 20:05:30.342 [pool-5-thread-1] INFO org.apache.bookkeeper.bookie.EntryLoggerAllocator - Created new entry log file data/standalone/bookkeeper0/current/1.log for logId 1. 20:05:40.046 [pulsar-web-69-5] INFO org.eclipse.jetty.server.RequestLog - 127.0.0.1 - - [08/Jul/2020:20:05:40 +0000] “GET /admin/v2/persistent/public/functions/coordinate/stats?getPreciseBacklog=false HTTP/1.1” 200 1278 “-” “Pulsar-Java-v2.6.0” 1
Issue Analytics
- State:
- Created 3 years ago
- Comments:12 (5 by maintainers)
Top GitHub Comments
Seems to be the same issue as https://github.com/apache/pulsar/issues/10515
I will try to put that together. Thanks!