PrefetchCount not work as expected
See original GitHub issueDescribe the bug
For some overloaded methods of IBus.Advanced.Consume
, the prefetchcount parameter is not work as expected
To Reproduce
string MQConnectionStrFormat =
"host={0};virtualHost={1};username={2};password={3};prefetchcount=10;publisherConfirms=true";
string conn = string.Format(MQConnectionStrFormat, "********", "****", "****","********");
var _bus = RabbitHutch.CreateBus(conn);
var queue=await _bus.Advanced.QueueDeclareAsync("testprefetch"
);
var exchange = _bus.Advanced.ExchangeDeclare("testprefetch", ExchangeType.Direct);
_=_bus.Advanced.Bind(exchange, queue, "testprefetch");
// publish test messages
Enumerable.Range(1, 20).ToList().ForEach( async i =>
{
var msg = new Message<string>("prefetch msg " + i);
await _bus.Advanced.PublishAsync(exchange, "testprefetch",false,msg);
});
then subscribe handlers in several ways
//work as expected , it consume 10 message per 5 seconds
_bus.Advanced.Consume(queue, async (data, mp, mri) =>
{
var message = Encoding.UTF8.GetString(data.ToArray());
Console.WriteLine(message);
await Task.Delay(5000);
return AckStrategies.Ack;
}, config =>
{
config.WithPrefetchCount(10);
});
// not work as expected, it consume 1 message per 5 second
_bus.Advanced.Consume(queue, async (data, mp, mri) =>
{
var message = Encoding.UTF8.GetString(data.ToArray());
Console.WriteLine(message);
Task.Delay(5000).Wait();
return AckStrategies.Ack;
}, config =>
{
config.WithPrefetchCount(10);
});
// not work as expected, it consume 1 message per 5 second
_bus.Advanced.Consume(queue, (data, mp, mri) =>
{
var message = Encoding.UTF8.GetString(data.ToArray());
Console.WriteLine(message);
Task.Delay(5000).Wait();
return Task.FromResult(AckStrategies.Ack);
}, config =>
{
config.WithPrefetchCount(10);
});
Only the first of the above methods can work as expected, which makes me very confused, Also, I dont know the difference between the prefetchcount parameter within the connection string and the WithPrefetchCount configured in the subscribe method, so i set both to 10…
I don’t know what caused the failure of the following two methods, which makes me very worried about unexpected problems in the program I wrote
So here again to summarize my question:
- What could cause the prefetchcount parameter to not work?
- wht is the difference between the prefetchcount parameter within the connection string and the WithPrefetchCount configured in the subscribe method?
Please complete the following information):
- EasyNetQ version: [e.g.
7.3.7
] - RabbitMQ version [e.g.
3.8.18
]
Issue Analytics
- State:
- Created 9 months ago
- Reactions:1
- Comments:9 (7 by maintainers)
Top Results From Across the Web
PubSub.SubscribeAsync prefetchCount does not work as ...
Hello,. Maybe I just missed some breaking change but it looks like in EasyNetQ 7 prefetchCount does not work the same way it...
Read more >django - Prefetch Related not working as expected
I'm developing an API with Django 1.11.11 and python 3.6.4. I have the following model: class Subsection(models.Model): genres = models.
Read more >Prefetch Azure Service Bus messages
If there are no messages available for delivery, the receive operation empties the buffer and then waits or blocks, as expected. Why is...
Read more >FAQ: How to Optimize the RabbitMQ Prefetch Count
Prefetching in RabbitMQ simply allows you to set a limit of the number of unacked (not handled) messages. There are two prefetch options ......
Read more >Prefetch_factor not working
Hi @ptrblck, I don't understand the exact use of the prefetch factor argument. Acc. to the docs, it is Number of batches loaded...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
I prefer to think about
PrefetchCount
as a count of messages which are prefetched from a queue without any knowledge of how many “threads” are used for processing.ConcurrencyLimit
should be used to set up count of threads for processing.Having said that, at the moment
ConcurrencyLimit
could be set only for all models of a connection, i.e. globally in case of EasyNetQ. RabbitMQ.Client needs to be patched to allow it on a per model basis.If you change code slightly, it will work as expected.
So the issue here is the following: if a message callback is executed synchronously(even if it is marked as async), then it could block dispatcher.
https://github.com/rabbitmq/rabbitmq-dotnet-client/blob/6.x/projects/RabbitMQ.Client/client/impl/AsyncConsumerWorkService.cs#L153