Get length/count of out queue for producer.
See original GitHub issueDescription
Really enjoying working with confluent-kafka-dotnet library, however for the life me, I can not work out how to get the length/count of out queue for a producer.
I could handle this myself by doing something like;
config.Add( "queue.buffering.max.messages", 1);
Then managing the messages to be sent using own queue, however I see very poor performance/throughput. I assume this is because it essentially negates, or at the least slows down the batch sending of messages.
Any wisdom would be very appreciated.
Issue Analytics
- State:
- Created 6 years ago
- Reactions:1
- Comments:17 (11 by maintainers)
Top Results From Across the Web
How do I find the Size of a Queue - ActiveMQ
Enqueue Count - the total number of messages sent to the queue since the ... The “size of a queue” is also explicitly...
Read more >Producer/consumer pattern with a fixed-size FIFO queue
The twist in this is that the queue needs to only hold a fixed number of items (strings, in my case). My application...
Read more >How to detect end of queue in a parallelized web crawler?
Let's say I want to program a parallelized web crawler which has a shared FIFO queue (multi consumer/multi producer). The queue only contains ......
Read more >GetQueueAttributes - Amazon Simple Queue Service
MessageRetentionPeriod – Returns the length of time, in seconds, for which Amazon SQS retains a message. When you change a queue's attributes, ...
Read more >Queue Length Limit
The maximum length of a queue can be limited to a set number of messages, or a set number of bytes (the total...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
A rough indication of the out queue length would be much preferred over having to maintain additional code outside of the client to track the length.
I place very high priority on not confusing users 😃.
As we just discussed, it’ll be a simple change to provide another method in librdkafka that doesn’t expose librdkafka internals, and I think we should do that and not expose this number.
note to anyone reading along:
rd_kafka_outq_len
is an appropriate value to compare to 0 in implementingFlush
, which is the reason it’s defined the way it is.