lambda: corrupt zip archive asset produced when using node v15.6

Update from the CDK team

When using local assets with the lambda module using the Code.fromAsset() API, deployment of new assets (during cdk deploy) will fail with the error - “Uploaded file must be a non-empty zip”

We have confirmed this issue manifests when using node engine v15.6. The root cause is being tracked by node - https://github.com/nodejs/node/issues/37027. For the time being, use a node engine < 15.6.

If you have encountered this issue, you need to do three things to resolve this -

1. Downgrade to a node version < 15.6. You can see what you’re current node version is by running node --version.
2. Delete the cached asset in the cdk.out/ directory. You could safely delete the entire directory, and the correct assets and templates will be re-generated during the next run of cdk synth or cdk deploy.
3. Delete the previously uploaded asset from the staging bucket. The S3 key of the asset will be prefixed with asset/ and have the hash of the asset. The hash of the asset can be found in your application’s CloudFormation template. The staging bucket can be obtained by running the following command -

 aws cloudformation describe-stack-resources --stack-name CDKToolkit --logical-resource-id StagingBucket  --query 'StackResources[].PhysicalResourceId'

---

Original issue filed below

When doing

const fillFn = new lambda.Function(this, "Zip2GzipSQSFill", {
    code: lambda.Code.fromAsset("lib/python"),
    runtime: lambda.Runtime.PYTHON_3_7,
    handler: "mylambda.handler"
    }
);

my file is here (relative to root of the CDK project): lib/python/mylambda.py

its contents:

def handler(event, context):
    print("HELLO AWS")

I get the error:

Uploaded file must be a non-empty zip (Service: AWSLambdaInternal; Status Code: 400; Error Code: InvalidParameterValueException; Request ID: fcfaf553-70d3-40b8-85d2-a15f6c3bcef0; Proxy: null)

Reproduction Steps

run cdk deploy

What did you expect to happen?

the Lambda is created in AWS

What actually happened?

An zip file is uploaded to the CDK staging bucket, there is a zip file there with my python file, but that file has no contents

Environment

• CDK CLI Version : 1.85.0 or 1.83.0
• Framework Version: 1.83.0
• Node.js Version: 15.6.0
• OS : MacOS
• Language (Version): TypeScript (4.1.3)

Other

I’m suspecting it’s a nodejs/library problem, in that some library is producing this invalid zip file but I have no evidence of this.

---

This is 🐛 Bug Report

[iliapolo]

Hi everyone, sharing my conclusions here as well:

• Working: crc32-stream >= 4.0.2 OR Node <= 15.5.0
• Non working: crc32-stream < 4.0.2 AND Node > 15.5.0

I know some folks have been reporting conflicting results, before we dive deeper into that, I wanted to clarify something about the subtleties of asset caches. It might explain some of this behavior.

Note that its not sufficient to remove only the asset your code is creating, but also the assets created by the framework.

For example, the BucketDeployment construct bundles its own asset (that contains the lambda code) inside the cdk.out directory. If you used a faulty combination and deployed the stack, the lambda code asset itself will be corrupted, and any subsequent deployment will suffer from the same problem, regardless of if you removed your own asset. To address this, you need to delete all assets that are included in your stack:

staging_bucket=$(aws cloudformation describe-stack-resources --stack-name CDKToolkit --logical-resource-id StagingBucket --query 'StackResources[].PhysicalResourceId' --output=text)
for asset in $(ls cdk.out | grep asset); do
  hash=$(echo ${asset} | cut -d'.' -f2)
  aws s3 rm s3://${staging_bucket}/assets/${hash}.zip
done

It’s also a good idea to just nuke the entire cdk.out directory instead of specific assets inside it.

I’ve created this reproduction repo that does all that for you and also allows specifying different versions of the relevant components. You can use it for sanity checks as well as plugging in your own stack.

[jmpalomar]

@MHacker9404 @machielg why aren’t you using an active LTS version of node.js?

Good question, brew installs 15.x by default.

Is it possible for aws-cdk to specify the compatible/supported version(s)? So it doesn’t install on node 15.x or gives a warning

You can do it: brew install node@14 && brew link node@14 --overwrite

Downgrading to node 14.15.4 and cleaning cdk.out and s3 zip files made the trick for me.