Version & Release Management for Lambda-Layer packages

For the initiative to reduce the size of the monolothic aws-cdk-lib package, we are separating out the lambda-layer-xyz modules to their own v2-compatible packages. The purpose of this issue is to make a decision about how to manage the versions and releases of those packages.

See https://github.com/aws/aws-cdk-rfcs/blob/madeline-k/reduce-module-size/text/0039-reduce-module-size.md for more context.

Proposed:

Release the v2-compatible @aws-cdk/lambda-layer-xyz packages on a separate cadence from aws-cdk-lib and each one in a separate cdklabs repo (cdklabs/awscdk-lambda-layer-xyz). All aws-cdk-lib APIs that are used by the lambda-layer packages are available in the first stable 2.0 release. So the @aws-cdk/lambda-layer-xyz packages can take a peer and dev dependency on aws-cdk-lib@2.0.0.

• The first release of cdklabs/awscdk-lambda-layer-xyz would include:
  • @aws-cdk/lambda-layer-xyz@2.0 with a peer & dev dependency on aws-cdk-lib@2.0
• Then, release 2.n of aws-cdk would include aws-cdk-lib@2.n which “dynamically loads” and has devDependencies on @aws-cdk/lambda-layer-xyz@2.0

Each lambda-layer dependency will have it’s own repo. And within each repo, we will have a release branch for each major or minor version of the tool that needs to be released in parallel. To create a new lambda layer package, with a new tool/dependency, we will need to create a new cdklabs repo for it. To add a new major version of an existing lambda layer package, a new branch in the existing repo will need to be created (a new branch for each minor version in the case of kubectl).

This option has the advantage of decoupling these packages from each other, and we could release updates to one without unnecessarily bumping the version for all the others.

Alternatives:

1. Create a single cdklabs repo to hold all of the lambda-layer packages. All @aws-cdk/lambda-layer-xyz packages would be released together using github workflows for each change pushed to this repo. This means to add a new one, or make a change to one, all of them would have a minor version release, or we would need to invent a unique solution for this repo to independently release each pacakge. While this could make our management of multiple repos easier in the long term, it will require inventing a unique solution for managing dependencies in this repo, and for independently versioning the packages that are released from this repo.

2. Use the current v2 release process. This would couple the aws-cdk-lib and @aws-cdk/lambda-layer-xyz releases together. With every v2 release, we would release a new version of the @aws-cdk/lambda-layer-xyz packages, even if there had been no changes.

   1. Release 2.n would include:
      1. aws-cdk-lib@2.n with no changes relevant to this topic
      2. @aws-cdk/lambda-layer-xyz@2.n with a dev and peer dependency on aws-cdk-lib@2.n
   2. Release 2.n+1 would include
      1. aws-cdk-lib@2.(n+1) which “dynamically loads” and has a devDependency on @aws-cdk-lambda-layer-xyz@2.n
      2. @aws-cdk/lambda-layer-xyz@2.(n+1) with a dev and peer dependency on aws-cdk-lib@2.(n+1)

[rix0rrr]

And we rely on Dependabot/npm-check-updates to upgrade to the latest version of each layer after it gets released, yeah?

I like.

[kaizencc]

I agree with the proposed solution. The process for using the current v2 release process is needlessly complicated, when we can leverage the existing release process in cdklabs repositories for a much cleaner solution. This is a clear use case for cdklabs imo. I think it will also be easier to maintain going forward. I can already imagine the hairy dependency issues that may pop up from using the current v2 release process, and all CDK team members would have to familiarize themselves with how lambda layers’ dependencies work. Contrast that with separate cdklabs repositories, which we already know how to maintain.