(eks): AwsAuth not updated, kubectl commands failing in CustomResources

Our cdk cluster recently started experiencing failures where our CustomResources are not authorized to call kubectl All kubectl commands result in the error error: You must be logged in to the server (Unauthorized).

Reproduction Steps

Create an AdminRole

const clusterAdmin = new Role(this, 'AdminRole', {
  roleName: 'EKSClusterAdmin',
  assumedBy: new AccountRootPrincipal(),
});

Create cluster, pass AdminRole to mastersRole parameter

const cluster = new Cluster(this, 'EKSCluster', {
  version: KubernetesVersion.V1_18,
  vpc: props.vpc,
  clusterName: 'EKSCluster',
  mastersRole: clusterAdmin,
  secretsEncryptionKey: secretsKey,
  defaultCapacity: 0,
  endpointAccess: EndpointAccess.PUBLIC_AND_PRIVATE,
});

Create a lambda with a kubectl layer

const lambda = new lambda.Function(this, id, {
  code: props.lambdaCode,
  functionName: props.lambdaFunctionName,
  handler: props.handler,
  runtime: lambda.Runtime.PYTHON_3_7,
  layers: [props.layerVersion],
  timeout: Duration.minutes(15),
  memorySize: 256,
  vpc: props.vpc,
  securityGroups: props.securityGroups,
});

1. Give lambda permission to assume the AdminRole

lambda.role.addToPolicy(
      PolicyStatement.fromJson({
        Effect: 'Allow',
        Action: ['sts:AssumeRole'],
        Resource: props.mastersRole.roleArn,
      })
    );

1. In a custom resource in this lambda, login using the admin role

aws eks update-kubeconfig --name EKSCluster --role-arn <AdminRoleArn> --kubeconfig /tmp/kubeconfig

Outputs Added new context arn:aws:eks:us-west-2:XXXXXXXXXXXX:cluster/EKSCluster to /tmp/kubeconfig

1. Make a kubectl call e.g. kubectl get psp --kubeconfig /tmp/kubeconfig

What did you expect to happen?

kubectl command succeeds

What actually happened?

error: You must be logged in to the server (Unauthorized)

Environment

• CDK CLI Version : 1.68.0

[robbierolin]

I think the built in provider didn’t exist when we first built our cluster or we just didn’t know about it. Seems like it would be a good option to switch to using it.

[github-actions[bot]]

This issue has not received a response in a while. If you want to keep this issue open, please leave a comment below and auto-close will be canceled.