[BUG] BlobLeaseClient retry behavior causes transient 409 failures on Release

Library name and version

Azure.Storage.Blobs 12.7.0

Describe the bug

Internal retry behavior in BloblLeaseClient.Release can expose callers to misleading 409 errors.

• The lease is released by this first try, but a timeout happens
• Release will automatically retry the release operation due to the timeout;
• The second try fails and encounters a 409 LeaseIdMismatchWithLeaseOperation error since the lease has already been released by the first request
• The Azure SDK throws a 409 exception

Report based on this issue.

Expected behavior

For a caller of the API, the LeaseIdMismatchWithLeaseOperation seems to indicate API misuse. Therefore, I think we should avoid throwing this in the case of transient failure followed by retry. One option could be to not throw at all in this scenario. Another (less desirable) option could be to throw, but use a different error so that callers can distinguish the particular outcome.

Actual behavior

See bug description

Reproduction Steps

Call Acquire[Async], then later call Release[Async] on the same BlobLeaseClient instance. This reproduces when Release[Async] hits a transient error.

I’m reporting on behalf of @rmt2021 who reported it here; perhaps they have a better repro.

Environment

@rmt2021 ?

[Jaxelr]

I’m having a similar scenario on a production issue but on the the blobClient.UploadAsync operation. I suppose the same workaround will apply to this, but there would be no way to discern real scenarios wherein the blob was existing prior to the upload operation.

[navba-MSFT]

Adding Service team to look into this.

@xgithubtriage please look into this once you get a chance and provide an update.