question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Reader: HasMessageAvailableAsync returns True but ReadNextAsync never returns when using StartMessageFromRollbackDuration

See original GitHub issue

I am not sure if this is a Pulsar issue or issue with Pulsar.Client. Hoping someone can help point me in the right direction!

The below function works perfectly when using StartMessageId(MessageId.Earliest). If I try to use StartMessageFromRollbackDuration like below, the HasMessageAvailableAsync() will return true but then the ReadNextAsync will block indefinitely, never returning a message. There is only one message in the queue at the time this runs, if that helps as well.

{
    TimeSpan rollbackDuration = DateTime.Now - startReadtime;
    await using IReader<byte[]> reader = await _readerBuilder
        .StartMessageFromRollbackDuration(rollbackDuration)
        //.StartMessageId(MessageId.Earliest)
        .CreateAsync();

        while (await reader.HasMessageAvailableAsync())
        {
            Message<byte[]> received = await reader.ReadNextAsync();

            if (CheckMessage(received, messageCheck))
            {
                return true;
            }
        }              

    return false;
}

Here are the logs from my Standalone Pulsar image running in Docker from the point of creating the read to attempting to read the next:

20:05:18.932 [pulsar-io-51-2] INFO org.apache.pulsar.broker.service.ServerCnx - [/192.168.176.8:33082] Subscribing on topic persistent://public/default/block-schedule-activity / reader-9b5a98d18f 20:05:18.934 [pulsar-io-51-2] INFO org.apache.pulsar.broker.service.persistent.PersistentTopic - [persistent://public/default/block-schedule-activity][reader-9b5a98d18f] Creating non-durable subscription at msg id null 20:05:18.934 [pulsar-io-51-2] INFO org.apache.bookkeeper.mledger.impl.NonDurableCursorImpl - [public/default/persistent/block-schedule-activity] Created non-durable cursor read-position=10:1 mark-delete-position=10:0 20:05:18.934 [pulsar-io-51-2] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/block-schedule-activity] Opened new cursor: NonDurableCursorImpl{ledger=public/default/persistent/block-schedule-activity, ackPos=10:0, readPos=10:1} 20:05:18.973 [bookkeeper-ml-workers-OrderedExecutor-1-0] INFO org.apache.pulsar.broker.service.persistent.PersistentMessageFinder - [persistent://public/default/block-schedule-activity][reader-9b5a98d18f] Found position 10:0 closest to provided timestamp 1594238715935 20:05:18.973 [bookkeeper-ml-workers-OrderedExecutor-1-0] INFO org.apache.pulsar.broker.service.persistent.PersistentSubscription - [persistent://public/default/block-schedule-activity][reader-9b5a98d18f] Successfully disconnected consumers from subscription, proceeding with cursor reset 20:05:18.973 [bookkeeper-ml-workers-OrderedExecutor-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [public/default/persistent/block-schedule-activity] Initiate reset position to 10:1 on cursor reader-9b5a98d18f 20:05:18.975 [bookkeeper-ml-workers-OrderedExecutor-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [public/default/persistent/block-schedule-activity] reset position to 10:1 before current read position 10:1 on cursor reader-9b5a98d18f 20:05:18.976 [bookkeeper-ml-workers-OrderedExecutor-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [public/default/persistent/block-schedule-activity-reader-9b5a98d18f] Rewind from 10:1 to 10:1 20:05:18.976 [bookkeeper-ml-workers-OrderedExecutor-1-0] INFO org.apache.pulsar.broker.service.persistent.PersistentTopic - [persistent://public/default/block-schedule-activity] There are no replicated subscriptions on the topic 20:05:18.976 [bookkeeper-ml-workers-OrderedExecutor-1-0] INFO org.apache.pulsar.broker.service.persistent.PersistentTopic - [persistent://public/default/block-schedule-activity][reader-9b5a98d18f] Created new subscription for 1 20:05:18.976 [bookkeeper-ml-workers-OrderedExecutor-1-0] INFO org.apache.pulsar.broker.service.ServerCnx - [/192.168.176.8:33082] Created subscription on topic persistent://public/default/block-schedule-activity / reader-9b5a98d18f 20:05:30.334 [SyncThread-7-1] INFO org.apache.bookkeeper.bookie.EntryLogManagerBase - Creating a new entry log file because current active log channel has not initialized yet 20:05:30.339 [SyncThread-7-1] INFO org.apache.bookkeeper.bookie.EntryLoggerAllocator - Created new entry log file data/standalone/bookkeeper0/current/0.log for logId 0. 20:05:30.342 [pool-5-thread-1] INFO org.apache.bookkeeper.bookie.EntryLoggerAllocator - Created new entry log file data/standalone/bookkeeper0/current/1.log for logId 1. 20:05:40.046 [pulsar-web-69-5] INFO org.eclipse.jetty.server.RequestLog - 127.0.0.1 - - [08/Jul/2020:20:05:40 +0000] “GET /admin/v2/persistent/public/functions/coordinate/stats?getPreciseBacklog=false HTTP/1.1” 200 1278 “-” “Pulsar-Java-v2.6.0” 1

Issue Analytics

  • State:open
  • Created 3 years ago
  • Comments:12 (5 by maintainers)

github_iconTop GitHub Comments

1reaction
Lanayxcommented, May 12, 2021

Seems to be the same issue as https://github.com/apache/pulsar/issues/10515

1reaction
JarrodJ83commented, Jul 9, 2020

I will try to put that together. Thanks!

Read more comments on GitHub >

github_iconTop Results From Across the Web

Issues · fsprojects/pulsar-client-dotnet
ReceiveAsync() use CancellationToken may cause a deadlock. ... Reader: HasMessageAvailableAsync returns True but ReadNextAsync never returns when using ...
Read more >
feedIterator.HasMoreResults is true but ...
As per my understanding, you are facing issue when testing command to Azure Cosmos DB you are seeing feedIterator.HasMoreResults is true and ...
Read more >
Cosmos DB call never returns with async call
I have a method in a WebApi that calls Cosmos via the SQL API that returns process configuration files but will never return....
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found