question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

[BUG] RenewMessageLockAsync succeeds but CompleteMessageAsync fails on when using EnableCrossEntityTransaction and processing a messages takes +10 minutes

See original GitHub issue

Library name and version

Azure.Messaging.ServiceBus 7.7.0

Describe the bug

Invoking RenewMessageLockAsync works, it updates the lease but eventually when processing is completed the call to CompleteMessageAsync fails.

Azure.Messaging.ServiceBus.ServiceBusException: The lock supplied is invalid. Either the lock expired, or the message has already been removed from the queue, or was received by a different receiver instance. (MessageLockLost)

This while LockedUntil on message object is minutes into the future.

What is important to note is that the ServiceBusReceiver instances are the same for invoking ReceiveMessageAsync and CompleteMessageAsync but not for RenewMessageLockAsync.

Interestingly on processing duration between 5 and 10-12 minutes things just work. Sometimes it also magically works for longer processing duration.

At the moment it seems when all methods share the same ServiceBusReceiver instance no MessageLockLost occurs when invoking CompleteMessageAsync.

Is it supported to invoke RenewMessageLockAsync on a different ServiceBusReceiver instance that invoked ReceiveMessageAsync ?

Expected behavior

I would expect that when RenewMessageLockAsync is invoked and it returns a message object with a valid LockedUntil that invoking CompleteMessageAsync will not result in a MessageLockLost, even if done so on a different ServiceBusReceiver instance.

Actual behavior

Invoking CompleteMessageAsync results in the following exception if the processing duration takes roughly 12 minutes or longer.

Azure.Messaging.ServiceBus.ServiceBusException: The lock supplied is invalid. Either the lock expired, or the message has already been removed from the queue, or was received by a different receiver instance. (MessageLockLost)

Reproduction Steps

Working on this, currently got a super hacky reproduction.

Environment

Tested with .NET 6 and .NET Framework 4.8

Issue Analytics

  • State:closed
  • Created a year ago
  • Comments:35 (18 by maintainers)

github_iconTop GitHub Comments

2reactions
JoshLove-msftcommented, May 10, 2022

@JoshLove-msft , Sorry for the late response.

I’m presenting Thomson Reuters. Right now we are using an older version of Azure Service Bus (ASB) client -“NServiceBus.Azure.Transports.WindowsAzureServiceBus”. We are migrating to .Net 5 and want to get rid of using .Net Framework. To be able to go further we need to start using newest version of ASB client. But it contains an issue with a message renewing that blocks us from the upgrade.

Could it be backported as a patch release? If yes, could you clarify approximate dates of that patch?

This was included in 7.8.0 - https://www.nuget.org/packages/Azure.Messaging.ServiceBus/7.8.0 which was just released today.

1reaction
JoshLove-msftcommented, Apr 22, 2022

Actually, I think you are right. The issue only occurs when setting EnableCrossEntityTransaction to true. I suspect this is related to the fact that we have a receiver and sender link on the same session. The sender link being closed is somehow causing the receive link to also be closed, presumably because they are sharing the session.

I think this is a bug in the SDK when using EnableCrossEntityTransactions. When closing the link, we also close the session, which we should not do in this case. Instead the session cleanup should happen only when the client is disposed. Thanks @ramonsmits for helping track this down!

Read more comments on GitHub >

github_iconTop Results From Across the Web

Azure Service Bus - Completemessageasync running slow
Hi, I have been doing some tests reading from a Azure Service Bus Queue using the library Azure.Messaging.ServiceBus in .NET 6.
Read more >
Azure service bus message lock and message completion
I'm using Azure service bus queue and I have a worker which takes a message and processes it for ~15 minutes whereas the...
Read more >
Receiving messages from Azure Service Bus in C# - Ciaran O ...
A quick demo showing how to receive messages from Service Bus using C# and .NET.
Read more >
Deep dive into Azure Service Bus messaging with the .NET ...
Before we get started with actual Azure Service Bus stuff, let's have a quick overview over the Azure messaging fleet together. So first...
Read more >
Azure Service Bus Essentials — Message Settlement with ...
await receiver. CompleteMessageAsync(receivedMessage); However, a request to settle a message may fail at the Service Bus side due to various ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found