[pipelines] Change to a single-source, single-build pipeline, deploy+approve model

Realign the API model of the library more towards what’s described in Automating Safe, Hands-off deployments.

This means we explicitly give up a bunch of freedom that CodePipeline allows, in order to make a simpler application deployment model. We’ll reject adding multiple source stages, multiple builds, and many types of actions in weird locations people might want to insert into the pipeline; if they want to do that, they should drop down to building a complete CodePipeline pipeline.

This means we do the following things:

• Create wrapper classes for GitHubSourceAction and others which have fewer required parameters. We’ll only ever have one source action, so no need to specify Artifacts.
• Same for Build Actions.
• Add the concept of an AppDeployment (name TBD); one pipeline stage can have 1 or more deploy AppDeployments; this is in order to do waves where we do multiple apps in parallel in one stage.
• An AppDeployment consists of two parts, that execute exactly in this order
  • 1 or more stack deployments, potentially with some ordering constraints between them.
  • 0 or more approval actions, that together form a sort of “approval workflow”. See the builders library article as well as the internal deployment concepts.
• There can be some passing of information between the stack deployments and the approval workflow (like the stack outputs now), but it only needs to be very limited in scope.
• The concept of an approval workflow step which can render to a CodeBuild project and a CodeBuild Action, or maybe a Lambda and a Lambda Action, a manual verification, etc.
  • Should allow cross-account and cross-region actions
• Design mostly to interfaces so we can swap out backends
• There is a use case for “additional actions” between Prepare and Execute. This would be an extension to the Builders Library API as it does not have this concept (see https://github.com/aws/aws-cdk/issues/11333)

---

I think the “action” model (with hooks for users to insert approvals) needs to look somewhat like this:

The “application approvals” should probably be configurable in direct style, it’s okay if the changeset approvals require implementing an interface of some sort (not ideal, but acceptable).

---

This might entail making changes to the CodePipeline L2 library; but only in order to make implementing CDK Pipelines easier, NOT to improve the API of the CodePipeline library. Some candidates:

• Move the codepipeline_actions.Action base class to the codepipeline library in order to make it easier to implement Actions.
• Make some Action properties like RunOrder and StackOutputs mutable, so we can change them after having created Actions.

---

I’m looking for compelling use cases that would require us to support multiple sources and/or multiple builds. Please supply them in this ticket if you have them.

On the deployment front, CDK pipelines will never do anything more than deploying CDK apps using CloudFormation, and running some limited set of validations.

[clareliguori]

Hey all! Thanks for pinging me @mrpackethead. In attempting to model how we do pipelines at Amazon in CDK, I think the number of source and build actions is less important than the order of stages in the pipeline and how the deployment actions are laid out.

Amazon pipelines have one single source stage (as seen in the diagram above), but that source stage typically brings in multiple sources (as seen here). For example, pretty much all pipelines internally at least bring in both the ‘primary’ source code repo and that repo’s dependency libraries in the source stage. A simple example to model in CDK would be bringing in the ‘primary’ source code (GitHub source action in CodePipeline terms) that will be built and deployed, plus a Docker base image (ECR source action in CodePipeline). Last time I checked, CodePipeline pipelines only have a single source stage for all source actions, so that already matches how we model pipelines internally.

After the source stage, we do all the build steps. Using the simple example I gave above, this would mean compiling the source code and packaging it into a Docker image that is pushed into ECR. Since CodePipeline already does the work for you of passing immutable artifacts between the build actions, I’m not especially opinionated about this being a single build action/stage vs split across multiple build actions/stages. Some build steps like replicating the docker image across ECR repos in multiple regions/accounts for deployment are useful to do in individual build actions (one per region/account). The area where we tend to be opinionated internally is simply doing ALL the build steps before ANY of the deployment steps, such that you have a consistent set of artifacts that will be deployed later in the pipeline.

When it comes to deployment stages, there is generally only one deployment action per AZ/region/account/any-other-unit in a wave’s stage in an internal pipeline. The key here for us is whether the operator can rollback a deployment in a single step. To use the example of DB migration scripts from @nbaillie above, in a single internal pipeline, we wouldn’t run a database migration script in a separate deployment action from deploying the microservice. Then we can’t rollback in a single step: the operator has to roll them back in a specific order manually, which can delay recovery and introduces human error. In that case, we would either a) split database migrations into a separate pipeline from the microservice’s pipeline, or b) combine them into a single deployment workflow action that deploys and rolls back in the correct order. We have an internal system for doing that, but @hoegertn’s suggestion of using a single CFN stack for both DB migration and microservice deployment is functionally equivalent in this case (using source dependency relationships in the stack, you would ensure CFN will deploy and roll back in the correct order).

Hope that helps!

[twitu]

Firstly CDK pipelines are awesome and I have really enjoyed using them. I want to add my thoughts on multi-source builds and why it will be a good addition.

A few examples have already been given above regarding polyglot projects. I also frequently encounter separate repos for infrastructure and application code in enterprises. There is a separation of responsibility between development and infrastructure/operations and permissions are also created along these lines.

Multi-source builds will be a great batteries included solution in such cases. The alternatives are workable but have fall short in terms of being seamless -

• Git submodules. They are problematic because they require additional commits to keep the monorepo in sync.
• Creating custom codepipelines which is feasible but a lot of extra effort only for the single source limitation.

The pace of development and deployment is a little slower in such cases, there is a hand-off between the development and infrastructure team. By having multi-source builds, the pipelines module will be flexible and unopinionated, allowing all kinds of teams to use it.

As a reverse question, what is the downside of supporting multiple sources in the pipeline?