Web PubSub SDK fails to refresh AAD access token with peak load
See original GitHub issue- Package Name: @azure/web-pubsub
- Package Version: 1.0.0
- Operating system: arm64-Darwin-21.6.0
- nodejs
- version: v16.15.0
- typescript
- version:
- Is the bug related to documentation in
- README.md
- source code documentation
- SDK API docs on https://docs.microsoft.com
Describe the bug
Our customer uses Web PubSub JavaScript SDK in their Azure Functions App with DefaultAzureCredential
for auth. They have found that the SDK sometimes keeps failing to auth to the Web PubSub service during peak load.
To Reproduce
Set up an HTTP trigger in Function App which calls any API of Web PubSub service client with DefaultAzureCredential
. Then call that HTTP trigger every 0.5s for more than one day. Then you will find the REST API calls of Web PubSub service client starts to fail and keeps failing.
In our repro project https://github.com/Y-Sindo/-AzureJSSdkAuthProblemRepro , We have customized the Web PubSub service SDK so that it can log the authentication header used to auth REST API call to prove that the access token is cached for more than one day and not refreshed, thus expires.
You can find detailed steps about how to use the repro project in its README.
Expected behavior The REST API call shouldn’t fail for access token invalid lifetime issue after running for one day.
Additional context Our customer encountered the issue in their product environment and want to get a solution as soon as possible.
Suspicion The invalid lifetime token problem can mitigate itself after some time in the production environment. But in our repro project, it didn’t mitigate by itself for more than one hour until I manually stopped the calls to the Http trigger of Functions. Thus, we suspect that the refresh of AAD access token is blocked when the function APP is busy with handling incoming requests. (Actually per 0.5s a request can trigger the problem, not a high concurrency)
Issue Analytics
- State:
- Created a year ago
- Comments:11 (8 by maintainers)
@Y-Sindo thank you for all your diving into this and my apologies that it seems to be a duplicate of an existing issue. I put up a PR that I think fixes it at #23232 but I need to set up a test environment to validate still.
@Y-Sindo this should be fixed in
@azure/identity@3.0.0
which released yesterday, please let me know if you are still seeing issues.