question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Unexpected behavior when seeking and receiving messages in the first batch of the topic

See original GitHub issue

In the dotnet client, when forcing 100 messages to be put into the same batch, and then seek the middle one like 50, it will start receiving from the first message. If batch sending is not enabled or forced to send one by one, it works normally.

This problem seems to only appear in the first batch of the topic.

Example codes:

const string serviceUrl = "pulsar://localhost:6650";

var client = await new PulsarClientBuilder()
    .ServiceUrl(serviceUrl)
    .BuildAsync();

var topicName = "public/default/topic-" + Guid.NewGuid().ToString("N");
logger.WriteLine(topicName);

var producer = await client.NewProducer()
    .Topic(topicName)
    .EnableBatching(true)
    .BatchingMaxPublishDelay(TimeSpan.FromSeconds(3))
    .CreateAsync();

var consumer = await client.NewConsumer()
    .Topic(topicName)
    .SubscriptionName("test-subscription")
    .SubscribeAsync();

// Act
int num = 100;
List<MessageId> msgs = new List<MessageId>(num);
List<Task<MessageId>> tasks = new List<Task<MessageId>>();

// When uncomment this, it will send the first batch, and the problem will not appear in the following batches.
// await producer.SendAsync(Encoding.UTF8.GetBytes("first-batch-message"));

for (int i = 0; i < num; i++)
{
    tasks.Add(producer.SendAsync(
        Encoding.UTF8.GetBytes($"[{i}]")));
}

await Task.WhenAll(tasks); // Force 100 messages to be sent in the same batch.
foreach (var task in tasks)
{
    msgs.Add(await task);
}

await consumer.SeekAsync(msgs[50]);

for (int i = 0; i < 10; i++)
{
    var reMsg = await consumer.ReceiveAsync();
    String msg = Encoding.UTF8.GetString(reMsg.Data);
    logger.WriteLine(msg);
}

Running these codes will output:

public/default/topic-215bd2ac57be47729b233b77695e2eb7
[0]
[1]
[2]
[3]
[4]
[5]
[6]
[7]
[8]
[9]

When uncommenting the line await producer.SendAsync(Encoding.UTF8.GetBytes("first-batch-message")); to send the first batch first. It will output the expected result. Expected output:

public/default/topic-215bd2ac57be47729b233b77695e2eb7
[51]
[52]
[53]
[54]
[55]
[56]
[57]
[58]
[59]
[60]

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:12 (5 by maintainers)

github_iconTop GitHub Comments

2reactions
Lanayxcommented, Feb 23, 2021

@RobertIndie Nice to hear! Please don’t forget to star the project if you find it useful 😃

0reactions
Lanayxcommented, Feb 26, 2021

@RobertIndie Yes, I’m aware of that 😃 That’s why I wrote “If message is received by Client between 5 and 6 steps”, which is before that logic happens

Read more comments on GitHub >

github_iconTop Results From Across the Web

Phishing Quiz | Federal Trade Commission
This is incorrect! In a phishing scam, you may get a message that looks like it's from someone you know and that asks...
Read more >
Screening for Behavioral Health Risk in Schools
Universal screening for behavioral and mental health issues can help with early identification of students who are at-risk or in need of intervention...
Read more >
Obsessive-Compulsive Disorder: When Unwanted Thoughts ...
People who are distressed by recurring, unwanted, and uncontrollable thoughts or who feel driven to repeat specific behaviors may have obsessive-compulsive ...
Read more >
Communication Skills - Start Here - Discover Our Top 100 ...
Effective communication isn't as common in business as it should be. Explore 100 tools and resources to help develop your communication skills at...
Read more >
The Surprising Power of Simply Asking Coworkers How ...
Assume positive intent: Start any conversation with your colleagues believing that those talking or listening mean well, especially when it ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found