[BUG] ServiceBus MessageReceiver: can't process more than prefetchCount messages simultaneously.

Describe the bug We are using ServiceBus to trigger long-running tasks processing. We can’t process more than prefetchCount messages simultaneously, because MessageReceiver.ReceiveAsync stops returning any messages.

Exception or Stack Trace N/A

To Reproduce Send N messages to ServiceBus. Create new MessageReceiver with options: receiveMode: ReceiveMode.PeekLock, prefetchCount: M, where M < N. Call MessageReceiver.ReceiveAsync in a loop and don’t complete any messages. Only M message will be returned in a total.

NOTE: MessageReceiver.ReceiveAsync will return new messages after LockToken expiration or completing message.

Code Snippet

        public async Task TestWithoutComplete()
        {
            int messageCount = 100;

            var queueClient = new QueueClient(ConnectionString, QueueName);
            var messagesToSend = Enumerable
                .Range(1, messageCount)
                .Select(i => new Message()).ToList();
            await queueClient.SendAsync(messagesToSend);

            var receiver = new MessageReceiver(ConnectionString, QueueName, receiveMode: ReceiveMode.PeekLock, prefetchCount: messageCount / 2);
            var receivedMessages = new List<Message>();

            while (messageCount != receivedMessages.Count)
            {
                var newMessages = await receiver.ReceiveAsync(messageCount);
                if (newMessages != null)
                {
                    receivedMessages.AddRange(newMessages);
                }
            }
        }

Expected behavior Receive message until ServiceBus Queue is empty.

Screenshots N/A

Setup (please complete the following information):

• OS: Windows 10 or Service Fabric
• IDE : Microsoft Visual Studio 2019
• NuGet package: Microsoft.Azure.ServiceBus 3.4.0

Additional context It was tested on different numbers of messages and prefetchCount.

Information Checklist Kindly make sure that you have added all the following information above and checkoff the required fields otherwise we will treat the issuer as an incomplete report

• Bug Description Added
• Repro Steps Added
• Setup information Added

[NikitaG]

@SeanFeldman, sorry if I wasn’t clear. I am not getting message from MessageReceiver though there are messages in broker. Example: If you send to the ServiceBus 1000 messages and then set receiver to prefetchCount: 100, you will be able to process only 100 messages simultaneously, when you complete some of messages you will be able to receive new messages.

I created repo to demonstrate issue: https://github.com/NikitaG/ServiceBusReproBug/blob/master/ServiceBusReproBug/Program.cs

If you run this application you can see, we are not receiving new messages until we complete previously received:

[2019-06-13T16:27:28]   New queue TestQueuefdcd0f37-4f12-430f-8221-66d0f46ea734 created.
[2019-06-13T16:27:29]   100 message(s) sent.
[2019-06-13T16:27:30]   New 10 message(s) received. Total unique messages received: 10. Messages in queue: 100.
[2019-06-13T16:27:35]   No messages received. Messages in queue: 100.
[2019-06-13T16:27:40]   No messages received. Messages in queue: 100.
[2019-06-13T16:27:45]   No messages received. Messages in queue: 100.
[2019-06-13T16:27:46]   10 message(s) completed. 
[2019-06-13T16:27:46]   New 6 message(s) received. Total unique messages received: 16. Messages in queue: 90.
[2019-06-13T16:27:51]   No messages received. Messages in queue: 90.
[2019-06-13T16:28:05]   No messages received. Messages in queue: 90.
[2019-06-13T16:28:05]   6 message(s) completed.
[2019-06-13T16:28:05]   New 6 message(s) received. Total unique messages received: 22. Messages in queue: 84.
[2019-06-13T16:28:10]   No messages received. Messages in queue: 84.

[axisc]

@NikitaG

I agree with @SeanFeldman recommendation. If you try to receive more than on the prefetch, you will face some delay as a round trip is needed to retrieve messages from the service bus namespace.

If you still have follow up questions, please open another issue.