question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Get length/count of out queue for producer.

See original GitHub issue

Description

Really enjoying working with confluent-kafka-dotnet library, however for the life me, I can not work out how to get the length/count of out queue for a producer.

I could handle this myself by doing something like;

config.Add( "queue.buffering.max.messages", 1);

Then managing the messages to be sent using own queue, however I see very poor performance/throughput. I assume this is because it essentially negates, or at the least slows down the batch sending of messages.

Any wisdom would be very appreciated.

Issue Analytics

  • State:open
  • Created 6 years ago
  • Reactions:1
  • Comments:17 (11 by maintainers)

github_iconTop GitHub Comments

2reactions
matth0x1commented, Oct 4, 2017

A rough indication of the out queue length would be much preferred over having to maintain additional code outside of the client to track the length.

1reaction
mhowlettcommented, Oct 4, 2017

I place very high priority on not confusing users 😃.

As we just discussed, it’ll be a simple change to provide another method in librdkafka that doesn’t expose librdkafka internals, and I think we should do that and not expose this number.

note to anyone reading along: rd_kafka_outq_len is an appropriate value to compare to 0 in implementing Flush, which is the reason it’s defined the way it is.

Read more comments on GitHub >

github_iconTop Results From Across the Web

How do I find the Size of a Queue - ActiveMQ
Enqueue Count - the total number of messages sent to the queue since the ... The “size of a queue” is also explicitly...
Read more >
Producer/consumer pattern with a fixed-size FIFO queue
The twist in this is that the queue needs to only hold a fixed number of items (strings, in my case). My application...
Read more >
How to detect end of queue in a parallelized web crawler?
Let's say I want to program a parallelized web crawler which has a shared FIFO queue (multi consumer/multi producer). The queue only contains ......
Read more >
GetQueueAttributes - Amazon Simple Queue Service
MessageRetentionPeriod – Returns the length of time, in seconds, for which Amazon SQS retains a message. When you change a queue's attributes, ...
Read more >
Queue Length Limit
The maximum length of a queue can be limited to a set number of messages, or a set number of bytes (the total...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found