question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

High latency inside ecs docker with latest versions(From V2.575.0) of aws sdk. It is working till V2.574.0

See original GitHub issue

Describe the bug A clear and concise description of what the bug is.

Is the issue in the browser/Node.js? Node.js

If on Node.js, are you running this on AWS Lambda? No Running on ecs docker Details of the browser/Node.js version Paste output of npx envinfo --browsers or node -v

SDK version number Example: v2.466.0

  • For browsers, the SDK version number is in the script tag
    src=“…/aws-sdk-2.466.0.min.js”
  • For Node.js, get SDK version by
    • running command npm list aws-sdk from your root directory
    • printing the output of AWS.VERSION in your code where AWS = require("aws-sdk"); aws-sdk@2.595.0

To Reproduce (observed behavior) Steps to reproduce the behavior (please share code or minimal repo) Install the V2.575.0 or Greater.

  1. Inside ec2 query request is taking ~60 ms
  2. Inside docker query request is taking ~4500ms Expected behavior In version 2.24.0 It is working fine in both ec2 and ecs container. It is giving results in ~60ms But in the latest versions(From From V2.575.0), It is taking ~4500ms inside the docker. Screenshots If applicable, add screenshots to help explain your problem.

Additional context Add any other context about the problem here.

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Comments:6 (3 by maintainers)

github_iconTop GitHub Comments

1reaction
chartrand22commented, Feb 20, 2020

We experience the same issue with SDK v2.575.0 and up. These bug fixes introduced this issue :

bugfix: IMDS: Signed IMDS workflow bugfix: IMDS: Type definitions update to signed IMDS workflow

Basically, this was done to tighten up security. (see IMDSv2) The updated flow requires requires a token to be obtained prior to being able to call any metadata endpoints.

From v2.575.0, the SDK is configured to default to the IMDSv2 workflow and, by default, will try three times (with a timeout of one second between attempts) to obtain the required token. If all three attempts fail, the SDK will then fall back to the IDMSv1 workflow, which in your case is successful. Once the credentials are obtained, your code is able to proceed and assume the requested role.

The workaround and/or solutions are :

  • use ECS task roles (probably the best solution)
  • look into disableFetchToken of the MetadataService (but requires handling refreshing of credentials)
  • allow extra hops on the EC2 instance metadata options : http-put-response-hop-limit
0reactions
lock[bot]commented, Mar 10, 2020

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs and link to relevant comments in this thread.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Troubleshoot high latency on a DynamoDB table - AWS
I'm seeing an increase in the response time for Amazon DynamoDB requests. Resolution. To troubleshoot high latency on your DynamoDB table, ...
Read more >
Troubleshoot common errors with API calls in Amazon ECS
Last updated: 2022-04-12. I want to troubleshoot common errors with API calls in Amazon Elastic Container Service (Amazon ECS).
Read more >
Class: AWS.ECS — AWS SDK for JavaScript
Describes the specified services running in your cluster. ... For example, if the computed desired count is 1.2, it rounds up to 2...
Read more >
ContainerDefinition Class | AWS SDK for .NET V2
This parameter specifies the minimum amount of CPU to reserve for a container, and containers share unallocated CPU units with other containers on...
Read more >
Manage Amazon ECS or Fargate Tasks with Step Functions
Step Functions can control certain AWS services directly from the Amazon States Language. For more information about working with AWS Step Functions and...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found