[FEATURE REQ] Support for safe batching with MessageSender

Is your feature request related to a problem? Please describe. As described in https://github.com/Azure/azure-service-bus-dotnet/pull/539

The client allows to send a collection of messages (SendAsync(IList<Message>)) which is problematic when there’s a large number of messages that would not fit into the maximum message size. As a result of that, an exception is thrown when the send operation is invoked.

Describe the solution you’d like I’d like to write the following code without worrying about an MessageSizeExceededException

await messageSender.SendAsync(bigListOfMessages);

But since there already a SendAsync method that takes IList<Message> one could name it SendChunkedAsync(), SendBatchAsync() or something else that doesn’t conflict.

I’m thinking that this method would use the current SendAsync(IList<Message>) method to send it’s chunks of Messages in order to pack them into as few AMPQ messages as possible.

Describe alternatives you’ve considered

1. I’ve tried to chunk the messages my self as described in this article but given that the Size property of the Message class only accounts for body size.

2. This is the my current solution from my understanding sends all the messages individually. Or at least it appears so on the when looking at the code.

var sendTasks = messages
    .Select(m => messageSender.SendAsync(m))
    .ToArray();
await Task.WaitAll(sendTasks);

[jsquire]

Have you considered using Azure.Messaging.ServiceBus, which recently went GA and is now considered the current generation library for Service Bus? It offers the ability to build a batch that is aware of size limits and that can be sent without the potential of exceeding the limit and triggering an exception.

For example:

// add the messages that we plan to send to a local queue
Queue<ServiceBusMessage> messages = new Queue<ServiceBusMessage>();
messages.Enqueue(new ServiceBusMessage("First message"));
messages.Enqueue(new ServiceBusMessage("Second message"));
messages.Enqueue(new ServiceBusMessage("Third message"));

// create a message batch that we can send
// total number of messages to be sent to the Service Bus queue
int messageCount = messages.Count;

// while all messages are not sent to the Service Bus queue
while (messages.Count > 0)
{
    // start a new batch
    using ServiceBusMessageBatch messageBatch = await sender.CreateMessageBatchAsync();

    // add the first message to the batch
    if (messageBatch.TryAddMessage(messages.Peek()))
    {
        // dequeue the message from the .NET queue once the message is added to the batch
        messages.Dequeue();
    }
    else
    {
        // if the first message can't fit, then it is too large for the batch
        throw new Exception($"Message {messageCount - messages.Count} is too large and cannot be sent.");
    }

    // add as many messages as possible to the current batch
    while (messages.Count > 0 && messageBatch.TryAddMessage(messages.Peek()))
    {
        // dequeue the message from the .NET queue as it has been added to the batch
        messages.Dequeue();
    }

    // now, send the batch
    await sender.SendMessagesAsync(messageBatch);

    // if there are any remaining messages in the .NET queue, the while loop repeats
}

More information on Azure.Messaging.ServiceBus can be found on its README and samples.

//cc: @JoshLove-msft

[ramya-rao-a]

Hello all,

The newer package Azure.Messaging.ServiceBus is available as of November 2020. While the older package Microsoft.Azure.ServiceBus will continue to receive critical bug fixes, we strongly encourage you to upgrade. Read the migration guide for more details.

Closing this issue as there are no plans to add the feature discussed here to the older Microsoft.Azure.ServiceBus package and @jsquire has provided pointers on how to use this feature in the Azure.Messaging.ServiceBus

Thanks for your patience!