Refresh token loop with eventhub managed identity

• UAMQP Package Version: 1.2.15
• Operating System: Linux
• Python Version: 3.9.2

Describe the bug When using with managed identity and eventhub, the producer client aparently get’s into a token refresh-loop. I don’t really know why this is happening, but the only explanation I can think of is that the Managed Identity implementation returns a cached token, with the same expiry, until it actually needs to be refreshed? (https://github.com/Azure/azure-sdk-for-python/blob/4b6911f2fb8d38f3edea80810cdab60b33c07c27/sdk/identity/azure-identity/azure/identity/_credentials/managed_identity.py#L202). The code still works, but it causes excessive logging around the time when refresh of token is required:

2021-03-07T14:50:13.829256117Z stderr F 2021-03-07 14:50:13,829 - azure.identity._credentials.default - INFO - DefaultAzureCredential acquired a token from ManagedIdentityCredential
2021-03-07T14:50:13.829235017Z stderr F 2021-03-07 14:50:13,829 - azure.identity._internal.decorators - INFO - ManagedIdentityCredential.get_token succeeded
2021-03-07T14:50:13.829122417Z stderr F 2021-03-07 14:50:13,828 - uamqp.authentication.cbs_auth - INFO - Token on connection b'EHProducer-811b7456-c441-4e89-a75b-9aec36cb7a4f' will expire soon - attempting to refresh.
2021-03-07T14:50:13.828887217Z stderr F 2021-03-07 14:50:13,828 - uamqp.c_uamqp - INFO - Token put complete with result: 1, status: 202, description: b'Accepted', connection: b'EHProducer-811b7456-c441-4e89-a75b-9aec36cb7a4f'
2021-03-07T14:50:13.824571313Z stderr F 2021-03-07 14:50:13,824 - azure.identity._credentials.default - INFO - DefaultAzureCredential acquired a token from ManagedIdentityCredential
2021-03-07T14:50:13.824523513Z stderr F 2021-03-07 14:50:13,824 - azure.identity._internal.decorators - INFO - ManagedIdentityCredential.get_token succeeded
2021-03-07T14:50:13.824455912Z stderr F 2021-03-07 14:50:13,824 - uamqp.authentication.cbs_auth - INFO - Token on connection b'EHProducer-811b7456-c441-4e89-a75b-9aec36cb7a4f' will expire soon - attempting to refresh.
2021-03-07T14:50:13.824120512Z stderr F 2021-03-07 14:50:13,823 - uamqp.c_uamqp - INFO - Token put complete with result: 1, status: 202, description: b'Accepted', connection: b'EHProducer-811b7456-c441-4e89-a75b-9aec36cb7a4f'
2021-03-07T14:50:13.82227951Z stderr F 2021-03-07 14:50:13,822 - azure.identity._credentials.default - INFO - DefaultAzureCredential acquired a token from ManagedIdentityCredential
2021-03-07T14:50:13.82224151Z stderr F 2021-03-07 14:50:13,822 - azure.identity._internal.decorators - INFO - ManagedIdentityCredential.get_token succeeded
2021-03-07T14:50:13.82217281Z stderr F 2021-03-07 14:50:13,822 - uamqp.authentication.cbs_auth - INFO - Token on connection b'EHProducer-811b7456-c441-4e89-a75b-9aec36cb7a4f' will expire soon - attempting to refresh.
2021-03-07T14:50:13.82203101Z stderr F 2021-03-07 14:50:13,821 - uamqp.c_uamqp - INFO - Token put complete with result: 1, status: 202, description: b'Accepted', connection: b'EHProducer-811b7456-c441-4e89-a75b-9aec36cb7a4f'
2021-03-07T14:50:13.819128307Z stderr F 2021-03-07 14:50:13,819 - azure.identity._credentials.default - INFO - DefaultAzureCredential acquired a token from ManagedIdentityCredential
2021-03-07T14:50:13.819080807Z stderr F 2021-03-07 14:50:13,818 - azure.identity._internal.decorators - INFO - ManagedIdentityCredential.get_token succeeded
2021-03-07T14:50:13.819007207Z stderr F 2021-03-07 14:50:13,818 - uamqp.authentication.cbs_auth - INFO - Token on connection b'EHProducer-811b7456-c441-4e89-a75b-9aec36cb7a4f' will expire soon - attempting to refresh.
2021-03-07T14:50:13.818849707Z stderr F 2021-03-07 14:50:13,818 - uamqp.c_uamqp - INFO - Token put complete with result: 1, status: 202, description: b'Accepted', connection: b'EHProducer-811b7456-c441-4e89-a75b-9aec36cb7a4f'
2021-03-07T14:50:13.817069905Z stderr F 2021-03-07 14:50:13,817 - azure.identity._credentials.default - INFO - DefaultAzureCredential acquired a token from ManagedIdentityCredential
2021-03-07T14:50:13.817033305Z stderr F 2021-03-07 14:50:13,816 - azure.identity._internal.decorators - INFO - ManagedIdentityCredential.get_token succeeded
2021-03-07T14:50:13.816944205Z stderr F 2021-03-07 14:50:13,816 - uamqp.authentication.cbs_auth - INFO - Token on connection b'EHProducer-811b7456-c441-4e89-a75b-9aec36cb7a4f' will expire soon - attempting to refresh.
2021-03-07T14:50:13.816841605Z stderr F 2021-03-07 14:50:13,816 - uamqp.c_uamqp - INFO - Token put complete with result: 1, status: 202, description: b'Accepted', connection: b'EHProducer-811b7456-c441-4e89-a75b-9aec36cb7a4f'
2021-03-07T14:50:13.814536002Z stderr F 2021-03-07 14:50:13,814 - azure.identity._credentials.default - INFO - DefaultAzureCredential acquired a token from ManagedIdentityCredential
2021-03-07T14:50:13.814459902Z stderr F 2021-03-07 14:50:13,814 - azure.identity._internal.decorators - INFO - ManagedIdentityCredential.get_token succeeded
2021-03-07T14:50:13.814389802Z stderr F 2021-03-07 14:50:13,814 - uamqp.authentication.cbs_auth - INFO - Token on connection b'EHProducer-811b7456-c441-4e89-a75b-9aec36cb7a4f' will expire soon - attempting to refresh.

It continues like this until a new token is aquired, producing heaps of logging output and cpu usage if the eventhub is receiving a lot of messages.

To Reproduce Steps to reproduce the behavior:

1. Set up Eventhub producer client with Managed Identity
2. Wait until expiry is near

Expected behavior Refresh should only happen once. Also, the managed identity will handle refresh of token internally, so there’s no need for this to happen at all.

I don’t know if this belongs here, or in the azure sdk repository, so feel free to move if appropriate.

[nilsga]

Any code creating a producer instance, and then wait for the expiry should be enough to reproduce. It’s within a AKS cluster, using pod identity, so it is a user assigned identity. I have disabled logging for this component now to avoid spamming, but I will enable and monitor to see at what interval this occurs.

[msftbot[bot]]

Hi, we’re sending this friendly reminder because we haven’t heard back from you in a while. We need more information about this issue to help address it. Please be sure to give us your input within the next 7 days. If we don’t hear back from you within 14 days of this comment the issue will be automatically closed. Thank you!