question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Reconsider the default aws:skipMetadataCheck setting

See original GitHub issue

In #1288 we disabled the metadata check in order to speed Pulumi up.

Unfortunately, this breaks Pulumi when running with an EC2 instance role, meaning lots of scenarios like self hosted CI/CD runners will not work.

It’s possible of course to fix this by setting the environment variable AWS_EC2_METADATA_DISABLED=false or pulumi config set aws:skipMetadataApiCheck false, but the it’s extremely difficult to figure that out unless you inherently know about this setting. This default setting seems to be violating the principal of least surprise and is causing a headache for those users who are setting Pulumi up in CI/CD.

To rectify this, we need to either:

  1. Provide concrete information back to the user about how to fix this. This might involve intercepting the AWS SDK calls and injecting information about why they’re getting this information
  2. Consider setting the default value back to false

it might be possible to intercept the error message, which currently looks a bit like this:

Error: NoCredentialProviders: no valid providers in chain
    caused by: EnvAccessKeyNotFound: failed to find credentials in the environment.
    SharedCredsLoad: failed to load profile, .
    EC2RoleRequestError: no EC2 instance role found
    caused by: RequestCanceled: EC2 IMDS access disabled via AWS_EC2_METADATA_DISABLED env var

But this comes from the AWS SDK, so we’d need to trap the error

Issue Analytics

  • State:open
  • Created 2 years ago
  • Reactions:15
  • Comments:10 (1 by maintainers)

github_iconTop GitHub Comments

2reactions
blarghmateycommented, Mar 29, 2022

This is still a problem. It would be great if we can get the default changed back, or at least fix the fact that the environment variable isn’t being picked up properly by the TF provider at the bottom of the chain.

0reactions
robotraptacommented, Nov 27, 2022

AFAICT this problem manifests with the unhelpful error message

    Error: failed to refresh cached credentials, no EC2 IMDS role found, operation error ec2imds: GetMetadata, access disabled to EC2 IMDS via client option, or "AWS_EC2_METADATA_DISABLED" environment variable

Dropping this here to help lost people like previous-me looking for an answer. The answer is:

pulumi config set aws:skipMetadataApiCheck false
Read more comments on GitHub >

github_iconTop Results From Across the Web

No results found

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found