[BUG] Azure KeyVault: SecretClient Token Expires

Describe the bug Hi all! We are using Azure KeyVault SDK in a Spring Boot + Kotlin project. On the start up of the application, we are setting up a SecretClient, that we are later using for fetching/setting customer related secrets.

However, some times the requests are failing, with the error “Token Expired” (See Stack Trace below)

Exception or Stack Trace

Suppressed: com.azure.core.exception.HttpResponseException: Status code 401, "{"error":{"code":"Unauthorized","message":"[TokenExpired] Error validating token: 'S2S12086'."}}"
2021-09-23 07:31:26
2021-09-23T05:31:26.446071136Z stdout F 		at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665) ~[kotlinx-coroutines-core-jvm-1.5.1.jar!/:na]
2021-09-23 07:31:26
2021-09-23T05:31:26.446065836Z stdout F 		at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678) ~[kotlinx-coroutines-core-jvm-1.5.1.jar!/:na]
2021-09-23 07:31:26
2021-09-23T05:31:26.446062036Z stdout F 		at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750) ~[kotlinx-coroutines-core-jvm-1.5.1.jar!/:na]
2021-09-23 07:31:26
2021-09-23T05:31:26.446052236Z stdout F 		at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571) ~[kotlinx-coroutines-core-jvm-1.5.1.jar!/:na]
2021-09-23 07:31:26
2021-09-23T05:31:26.446048536Z stdout F 		at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106) ~[kotlinx-coroutines-core-jvm-1.5.1.jar!/:na]
2021-09-23 07:31:26
2021-09-23T05:31:26.446045036Z stdout F 		at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) ~[kotlin-stdlib-1.5.10.jar!/:1.5.10-release-890 (1.5.10)]
2021-09-23 07:31:26
2021-09-23T05:31:26.446041636Z stdout F 		at net.grandcentrix.cellulariot.cellularhub.upstream.iothub.services.CloudConfigurationSecretManager$deleteSecrets$1$1$1.invokeSuspend(CloudConfigurationSecretManager.kt:132) ~[classes!/:na]
2021-09-23 07:31:26
2021-09-23T05:31:26.446038036Z stdout F 		at net.grandcentrix.cellulariot.cellularhub.utils.secretstore.AzureSecretStore.deleteSecret(SecretStore.kt:66) ~[classes!/:na]
2021-09-23 07:31:26
2021-09-23T05:31:26.446034036Z stdout F 		at net.grandcentrix.cellulariot.cellularhub.utils.secretstore.AzureSecretStore.setSecret(SecretStore.kt:28) ~[classes!/:na]
2021-09-23 07:31:26
2021-09-23T05:31:26.446029636Z stdout F 		at com.azure.security.keyvault.secrets.SecretClient.setSecret(SecretClient.java:97) ~[azure-security-keyvault-secrets-4.3.3.jar!/:4.3.3]
2021-09-23 07:31:26
2021-09-23T05:31:26.446026236Z stdout F 		at com.azure.security.keyvault.secrets.SecretClient.setSecretWithResponse(SecretClient.java:118) ~[azure-security-keyvault-secrets-4.3.3.jar!/:4.3.3]
2021-09-23 07:31:26
2021-09-23T05:31:26.446022635Z stdout F 		at reactor.core.publisher.Mono.block(Mono.java:1704) ~[reactor-core-3.4.9.jar!/:3.4.9]
2021-09-23 07:31:26
2021-09-23T05:31:26.446019035Z stdout F 		at reactor.core.publisher.BlockingSingleSubscriber.blockingGet(BlockingSingleSubscriber.java:99) ~[reactor-core-3.4.9.jar!/:3.4.9]

To Reproduce Happens automatically after the application has been running for a long time.

Expected behavior The SecretClient is using the KeyVaultCredentialPolicy. I’d expect the Secret Client to refresh the token automatically without the need to re-create the Secret Client. Is there perhaps some additional policy that I’m missing for this behavior?

Setup (please complete the following information):

• OS: Ubuntu
• Library/Libraries: com.azure:azure-security-keyvault-secrets:4.3.2
• Java version: 11
• Kotlin version: 1.5.10
• App Server/Environment: AKS
• Frameworks: Spring Boot

Additional context Add any other context about the problem here.

Information Checklist Kindly make sure that you have added all the following information above and checkoff the required fields otherwise we will treat the issuer as an incomplete report

• Bug Description Added
• Repro Steps Added
• Setup information Added

[kgiouzakov]

Hi @vcolin7 , we did a workaround for now, where we are reopening the client and retrying once in case the unauthorised exception occurs. I’ll make sure to upgrade the version and monitor the system in case the error still occurs. Thanks a lot 👍🏼

[vcolin7]

Hi @kgiouzakov, we recently released new versions of the Key Vault libraries that include changes to how the KeyVaultCredentialPolicy handles caching and refreshing tokens, would you mind trying using version 4.3.4 to see if the issue persists?