admin getMessageById vs api seek+readnext Performance difference

Performance of Regular org.apache.pulsar.client.api, is about 5x slower then admin org.apache.pulsar.client.admin to fetch single messages that were already consumend.

I am implementing a reader that needs to fetch 1 document at a time from bookeeper in a non sequential way.

Using a regular Reader to fetch a single document it takes around 119ms to fetch one message, while using the admin api it takes around 20ms is there any reason for this disparity?

I fetched 1000 messages using both methods to get those average times:

Reader API:

        PulsarClient client = PulsarClient.builder()
                .serviceUrl("pulsar://localhost:6650")
                .build();

        Reader reader = client.newReader()
                .topic("persistent://apache/pulsar/baby-yoda").startMessageId(MessageId.earliest)
                .startMessageIdInclusive()
                .receiverQueueSize(1) //only fetches a single document
                .create();        

        MessageId messageId= //somemessageid

        int counter=0;

        while (counter<1000) {

            reader.seek(messageId);
            Message document = reader.readNext();

            counter++;
        }

Admin API

PulsarAdmin admin =  PulsarAdmin.builder().serviceHttpUrl("http://localhost:8080").build();

        
        Message m = null;

        byte[] msg;

while (counter<1000) {
          
        //topic ledger entry
        try {
            msg = admin.topics().getMessageById("testtopic3",4,982).getData();
        } catch (PulsarAdminException e) {
            e.printStackTrace();
        }
  counter++;
        }

[codelipenghui]

The read API need to create an internal consumer to subscribe to the topic and send flow permits to the broker, then the broker read data from bookkeeper or tailing cache(batch read), but the admin API is reading the data from bookkeeper or tailing cache directly, and only read one message.

[eolivelli]

@DonHaul

Is blobit designed for realtime and reliability features?

yes it is, most notably it was initially designed to read the objects as soon as it has been written by the writer (it is using some feature in BK that is not used in Pulsar, like readUnconfirmed, in order to reduce latency. feel free to open a ticket with your use case on the issue tracker on github.

cc @nicoloboschi @diegosalvi @dianacle