Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Slow requests to get credentials from InstanceProfileCredentialsProvider

See original GitHub issue

I’ve noticed that the first request using SqsAsyncClient takes around 5000ms. This seems to be caused by InstanceProfileCredentialsProvider. The slow request seems to re-occur roughly every hour as the credential cache expires.

Stack trace of where 99% of time was spent in a slow request.

…net.www.protocol.http.HttpURLConnection.getInputStream (Unknown Source)
             java.net.HttpURLConnection.getResponseCode (Unknown Source)
…on.awssdk.regions.util.HttpResourcesUtils.readResource (HttpResourcesUtils.java:114)
…redentials.InstanceProfileCredentialsProvider.getToken (InstanceProfileCredentialsProvider.java:91)
…fileCredentialsProvider.getCredentialsEndpointProvider (InstanceProfileCredentialsProvider.java:76)
…credentials.HttpCredentialsProvider.refreshCredentials (HttpCredentialsProvider.java:74)
….amazon.awssdk.utils.cache.CachedSupplier.refreshCache (CachedSupplier.java:132)
  software.amazon.awssdk.utils.cache.CachedSupplier.get (CachedSupplier.java:89)
                                 java.util.Optional.map (Unknown Source)
…credentials.HttpCredentialsProvider.resolveCredentials (HttpCredentialsProvider.java:146)
…entials.AwsCredentialsProviderChain.resolveCredentials (AwsCredentialsProviderChain.java:91)
…internal.LazyAwsCredentialsProvider.resolveCredentials (LazyAwsCredentialsProvider.java:52)
…dentials.DefaultCredentialsProvider.resolveCredentials (DefaultCredentialsProvider.java:100)
…t.handler.AwsClientHandlerUtils.createExecutionContext (AwsClientHandlerUtils.java:71)
…t.handler.AwsAsyncClientHandler.createExecutionContext (AwsAsyncClientHandler.java:64)
software.amazon.awssdk.awscore.client.handler.AwsAsyncClientHandler.createExecutionContext…sdk.core.client.handler.BaseAsyncClientHandler.execute (BaseAsyncClientHandler.java:63)
…k.awscore.client.handler.AwsAsyncClientHandler.execute (AwsAsyncClientHandler.java:51)
….awssdk.services.sqs.DefaultSqsAsyncClient.sendMessage (DefaultSqsAsyncClient.java:1271)

I noticed that there is a similar issue for v1 of the java sdk. I am running this code from within a docker container, could my issue be related? https://github.com/aws/aws-sdk-java/issues/2171

Any thoughts would be awesome, thanks!

Issue Analytics

State:
Created 4 years ago
Reactions:5
Comments:11 (3 by maintainers)

Top GitHub Comments

3reactions

ffeltrinellicommented, Apr 8, 2022

Hi! I talked about this issue and described our custom solution in this article.

1reaction

ffeltrinellicommented, Feb 9, 2021

Hi! I’m also interested in this one. I’m having intermittent latency issues, reaching the non-configurable 1 sec read timeout multiple times per day, while the SDK is fetching credentials from IMDS. Not being able to configure a timeout shorter than 1 sec is problematic: my application is latency sensitive.

Top Results From Across the Web

Service Slow to Retrieve AWS Credentials - Medium

In our case the provider was InstanceProfileCredentialsProvider . This provider retrieves credentials by doing a network call to Amazon EC2 Instance Metadata ...

Developers - Slow requests to get credentials from ... - Bountysource

Slow requests to get credentials from InstanceProfileCredentialsProvider. ... I've noticed that the first request using SqsAsyncClient takes around 5000ms.

InstanceProfileCredentialsProvider (AWS SDK for Java

Spins up a new thread to refresh the credentials asynchronously. AWSCredentials · getCredentials(). Returns AWSCredentials which the caller can use to authorize ...

AWS SDK Credentials Provider slow at start up - Reddit

I have a local issue where the AWS SDK (Java) is invoking http://169.254.170 . 2 at startup and the timeout is slow. Every...

Unable to load AWS credentials on EC2 Instance

I have an EC2 Instance with Amazon Linux AMI on AWS: created user at AWS Identity and Access Management (IAM), granted AmazonSESFullAccess role ......