(eks): cluster only trusts cloudformation execution role with modern synthesis

General Issue

Can’t use CLI credentials when deploying stacks in CDK v2, causing issues with default EKS RBAC

The Question

There doesn’t seem to be a way to fall back to the old behaviour of using the CLI credentials to deploy CF stacks rather than the execution role that CDK Boostrap creates.

This is an issue for EKS, where new clusters are created by default with the creator’s role being the only one with master permissions. If you’re using CfnCluster rather than the custom resource based provider, this means that you can’t then fix the RBAC after the stack has finished creating, as the only entity that the cluster “trusts” is the cloudformation-only deploy role.

I would prefer not to use the CDK custom resource based provider.

CDK CLI Version

2.0.0-rc.23 (build 1e54fb9)

Framework Version

No response

Node.js Version

No response

OS

No response

Language

Typescript

Language Version

No response

Other information

No response

[mipearson]

@samjarrett is correct.

There’s three issues here:

(a) The first is that the documentation of BootstraplessStackSynthesizer is incorrect: it states that it’ll use the CLI credentials, but does not, as there’s a ||= in the code of the class BootstraplessStackSynthesizer is extending that reverts to using the created role as a default. (edit: and most importantly there’s no way to force it to use the CLI credentials without cooking up your own synthesizer, as in my example)

(b) Creating an EKS cluster via CfnCluster using this created role means the cluster can’t be accessed, as there’s no way to set up the master role mapping. There may be other similar issues with other resources & people’s expectations that the CLI role/user is the one used.

© This decision overall has some security implications that may be surprising, as @samjarrett pointed out. They were surprising to me! On the other hand, it’s a nice workaround to “oops my assumed role timed out in the middle of my big stack apply”.

[stevehodgkiss]

I ran into this issue upgrading to CDK 2 today. I realised due a PutKeyPolicy failure, which was due to the cfn-exec role not being on the key’s admin list (whereas the role provided via aws env vars credentials is).

There are other scenarios where we’d like to be using the role provided via env credentials, and not a cfn-exec role with extremely wide permissions. For example, we typically run CFN deploys with roles that don’t allow data destruction (to prevent accidental RDS “replacements” etc). To perform those actions a separate role needs to be assumed in order to make the action intentional (i.e aws-vault exec my-account-data-destroyer -- yarn cdk deploy ...). It’s not possible to use roles like this anymore if cdk deploy can’t be configured to use credentials from the environment.

It would be great if the use of an assumed cloudFormationExecutionRole could be disabled somehow (i.e. via cdk.json), so that deploys behave similar to CDK 1 and use the provided credentials in ENV when available.

This is a workaround for DefaultStackSynthesizer that avoids having to replace the synthesize function (using 2.0.0-rc.27):

class DefaultStackSynthesizerWithoutDefaultCloudFormationExecutionRole extends DefaultStackSynthesizer {
  protected emitStackArtifact(stack: Stack, session: ISynthesisSession, options: SynthesizeStackArtifactOptions = {}) {
    super.emitStackArtifact(stack, session, {
      ...options,
      cloudFormationExecutionRoleArn: undefined,
    });
  }
};

// Then make sure it's used in every cdk.Stack:
export class MyBaseCdkStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props: cdk.StackProps) {
    super(scope, id, {
      ...props,
      synthesizer: new DefaultStackSynthesizerWithoutDefaultCloudFormationExecutionRole(),
    });
  }
}

Using the LegacyStackSynthesizer is also an option. I added "@aws-cdk/core:newStyleStackSynthesis": false to cdk.json, and it requires the new CDKToolkit stack from CDK v2 bootstrap, and works as expected using the provided role credentials in ENV.