[s3-deployment] new style synthesis toolkit can not deploy existing app(without "@aws-cdk/core:newStyleStackSynthesis": true) with s3-deployment resources

Following below guide of cdk pipeline to upgrade cdk-toolkit stack to support new style synthesis,

https://aws.amazon.com/blogs/developer/cdk-pipelines-continuous-delivery-for-aws-cdk-applications/

Deploying an application with s3-deploy failed on error without useful information.

Do you wish to deploy these changes (y/n)? y
S3DeploymentOnNewStyleAssetsStack: deploying...
[0%] start: Publishing 4184245adc1f2ed71e1f0ae5719f8fd7f34324b750f1bf06b2fb5cf1f4014f50:current
[50%] success: Published 4184245adc1f2ed71e1f0ae5719f8fd7f34324b750f1bf06b2fb5cf1f4014f50:current
[50%] start: Publishing 28787f136b08be19118211a54b6d35fc304edd6b8cd350ac678b0599dc2ba3ed:current
[100%] success: Published 28787f136b08be19118211a54b6d35fc304edd6b8cd350ac678b0599dc2ba3ed:current
S3DeploymentOnNewStyleAssetsStack: creating CloudFormation changeset...
9:50:25 PM | CREATE_FAILED        | Custom::CDKBucketDeployment | WorkerConfFileDepl...omResource426C58D0
Failed to create resource. Command '['python3', '/var/task/aws', 's3', 'cp', 's3://cdk-hnb659fds-assets-034068960621-cn-northwest-1/assets/287
87f136b08be19118211a54b6d35fc304edd6b8cd350ac678b0599dc2ba3ed.zip', '/tmp/tmpn4sybwmm/3ad9f6c2-d3e1-4f43-9d1c-625d1ec2fa1a']' returned non-zer
o exit status 1.

	new CustomResource (/Users/mengxinz/tmp/s3-deployment-on-new-style-assets/node_modules/@aws-cdk/core/lib/custom-resource.ts:115:21)
	\_ new BucketDeployment (/Users/mengxinz/tmp/s3-deployment-on-new-style-assets/node_modules/@aws-cdk/aws-s3-deployment/lib/bucket-deployment.t
s:201:5)

Reproduction Steps

1. Update cdk.json like below,

"context": {
    "@aws-cdk/core:newStyleStackSynthesis": true,
    "@aws-cdk/core:enableStackNameDuplicates": "true",
    "aws-cdk:enableDiffNoFail": "true"
  }

1. bootstrap CDKToolkit like below,

npx cdk bootstrap --cloudformation-execution-policies arn:aws:iam::aws:policy/AdministratorAccess

1. Remove newStyleStackSynthesis property from cdk.json.
2. Deploying stack like below,

import * as cdk from '@aws-cdk/core';
import s3 = require('@aws-cdk/aws-s3');
import s3deploy = require('@aws-cdk/aws-s3-deployment');
import path = require('path');

export class S3DeploymentOnNewStyleAssetsStack extends cdk.Stack {
  constructor(scope: cdk.Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    // The code that defines your stack goes here
    const assetBucket = new s3.Bucket(this, 'TestBucket');

    const confPrefix = 'tunasync/worker/';
    const tmpOutput = path.join(__dirname, `./files`);
    const confFileDeployment = new s3deploy.BucketDeployment(this, 'WorkerConfFileDeployments', {
      sources: [s3deploy.Source.asset(tmpOutput)],
      destinationBucket: assetBucket,
      destinationKeyPrefix: confPrefix, // optional prefix in destination bucket
      prune: false,
      retainOnDelete: false,
    });
  }
}

Workaround:

Bootstrap a new cdk toolkit stack via option --toolkit-stack-name toolkit-version-0, then deploy above app via this toolkit stack.

Error Log

Environment

• CLI Version : 1.56.0
• Framework Version:
• Node.js Version:
• OS :
• Language (Version):

Other

---

This is 🐛 Bug Report

[iliapolo]

Indeed, interacting with the new bootstrap bucket requires having @aws-cdk/core:newStyleStackSynthesis: true during stack synthesis. The details are explained here: https://github.com/aws/aws-cdk/issues/9607.

If you cannot redeploy your resources with the new style synthesis, you need to add the following permission to bucket deployment role:

{
        Action: ['kms:Decrypt', 'kms:DescribeKey'],
        Effect: 'Allow',
        Resource: ['*'],
   }

Resolving in favor of https://github.com/aws/aws-cdk/issues/9607.

[jonny-rimek]

I’m running in the same problem: https://github.com/aws/aws-cdk/issues/9656