DeployCodeBaseCustomResource18AF4C97 failed

❓ General Issue

Same code works fine couple weeks before, but without any code change in the few weeks, it starts to fail every time when I tried to do cdk deploy. The error reported is from DeployCodeBaseCustomResource18AF4C97. Seems like the underlying aws-cli command failed for some reason.

 47/49 | 9:47:38 AM | CREATE_FAILED        | Custom::CDKBucketDeployment           | DeployCodeBase/CustomResource/Default (DeployCodeBaseCustomResource18AF4C97) Received response status [FAILED] from custom resource. Message returned: Command '['/opt/awscli/aws', 's3', 'cp', 's3://cdktoolkit-stagingbucket-574qtme5jv7w/assets/0b887894f0ecc543780dbe893f7115fc6ee8a5fde4ff9b9379406101a783a932.zip', '/tmp/tmplgzi9ntg/6f22d812-f3c1-434d-914d-585395b11860']' returned non-zero exit status 1. (RequestId: 852b0525-3088-4246-984e-221c86e5958f)

We used CDK for nodejs at version 1.103.0. The only related code I can think of is from @aws-cdk/aws-s3

        const code_bucket = new s3.Bucket(this, 'CodeBucket', {
            // removalPolicy: cdk.RemovalPolicy.DESTROY,
            // autoDeleteObjects: true,
        });
        new s3deploy.BucketDeployment(this, 'DeployCodeBase', {
            sources: [s3deploy.Source.asset('../benchmarks-stack')],
            destinationBucket: code_bucket,
        });

The whole CDK code can be found from https://github.com/awslabs/aws-c-s3/tree/main/benchmarks/dashboard-stack

Not sure it’s bug from CDK or something in our code base.

The Question

Environment

• CDK CLI Version: 1.103.0
• Module Version: 1.103.0
• Node.js Version: v16.1.0
• OS: macOS
• Language (Version): TypeScript

Other information

[peterwoodworth]

I suspect a re-bootstrap would solve this issue. I just cloned the repo and successfully deployed 😄

I’ll post guidance on re-bootstrapping in a bit

[omishagupta]

I was using KMS to upload the folder since I was uploading in encrypted bucket and was getting same error, while memory limit could have been one of the issues, it turned out despite of running the scripts as admin, the role lambda is using needs to be whitelisted on KMS too. Did that by using following snippet.

// Mention role to be used by bucket deployment
    new s3deploy.BucketDeployment(this, 'DeployWebsite', {
      sources: [s3deploy.Source.asset('../test')],
      destinationBucket: bucket,
      destinationKeyPrefix: props.folder,
      serverSideEncryption: s3deploy.ServerSideEncryption.AWS_KMS,
      serverSideEncryptionAwsKmsKeyId: props.key.keyArn,
      memoryLimit: 256, // as suggested here
      role: iam.Role.fromRoleArn(this, 'custom-role', <role-arn>)
    });
    
---
// Whitelist the role here
        const role = iam.Role.fromRoleArn(this, <construct-name>, <role-arn>)
        this.kmsKey.grantEncryptDecrypt(role)