[@aws-cdk/pipelines] Support multiple synthActions

I think by allowing multiple synth actions, and allowing us to control which stacks get synthesized when, we can have more elaborate deployment behavior.

(I just realized this may already be possible, I explain in the “Other” section why I’m uncertain)

Use Case

I’m hoping there’s another way to think about this or approach this.

The problem is two stacks, where stack B has an asset of a directory that is generated in a separate directory by an out-of-band process (npm run build) requiring cloudformation outputs from deployed stack A, because the values get substituted into the files upon generation. This is problematic because synthesis time is when the files get looked for and copied to the cloud assembly directory, so normally I would have to synthesize the first, deploy it, run npm run build, then synthesize and deploy the second.

Imagine the following stacks:

• Api with an AppSync resource and a CfnOutput of that resource’s GraphQL endpoint URL (AFAIK known only after deploying)

• Web with an S3 CloudFront distribution, using s3-deployment to add a directory as an asset and deploy it to an S3 bucket and invalidate the CloudFront cache

  • The key thing is that the files that Web bundles need to be generated by webpack via npm run build before bundling.
  • Equally as important, the GraphQL endpoint URL needs to be provided as an environment variable GRAPHQL_URL when invoking npm run build so that webpack substitutes a placeholder in files with the actual value.

In other words, the dependency/sequence is something like this:

1. Deploy Api stack, populates CfnOutputs like GRAPHQL_URL

2. cd web/ && GRAPHQL_URL="READ_FROM_CLOUDFORMATION_OUTPUT" npm run build

   This generates files at web/build/ with GRAPHQL_URL substituted in the output files

3. Deploy Web stack, which has web/build/ as an asset (for S3)

The problem with just slapping things together into a pipeline in the status quo is that the synthAction synthesizes all stacks at once.

This won’t work in this setup because doing so will “asset”-ize (?) Web’s files, even though they don’t exist yet since npm run build hasn’t run.

That can be be resolved by modifying synthAction’s synth command to run npm run build before the synth, but the problem is that the npm run build wouldn’t be appropriate here yet because Api hasn’t been deployed, so we wouldn’t know the GRAPHQL_URL yet.

We can go further and modify the synth command to synth Api only, and not synthesize Web (unless #9792 means that it’ll be synthesized anyway, but I probably misunderstood), so we can wait for Api to be deployed, then npm run build with the CfnOutput, but then how do we synthesize Web and deploy it? Now we’re left to our own devices, missing out on the pipeline machinery.

Proposed Solution

I honestly don’t know if this would fix the problem, but I just figure that if we can specify a synthAction anywhere like we can with other actions, then we can arrange them however we see fit. This way, I would have:

1. a synthAction for Api
2. ensure Api deploys (I don’t know if I need an explicit DeployCdkStackAction or if this is implicit)
3. a SimpleShellAction for npm run build of Web’s assets, since we can now read Api’s deployed CloudFormation outputs
4. finally another synthAction for Web now that the files have been generated

Other

Am I going about it the wrong way? Again, the issue fundamentally is two stacks where one has an asset requiring out-of-band generation that itself requires reading the CloudFormation outputs of the deployed stack, keeping in mind that synthesis-time is when CDK looks for and expects the assets.

I do see that SimpleSynthAction is just an IAction, but does that mean I really can just place it anywhere I want? The reason I’m not sure is because synthAction appears to be a specific attribute in CdkPipelineProps, so I’m not sure if simply adding it anywhere more than once would have the intended effect. If so, then I suppose I can just modify the CdkPipelineProps.synthAction’s synthCommand attribute to just synthesize Api, and then other actions, and then my own constructed SimpleSynthAction this time for Web?

• 👋 I may be able to implement this feature request
• ⚠️ This feature might incur a breaking change

---

This is a 🚀 Feature Request

[maciekbb]

@blaenk I wrote a post on how to solve it https://maciekbb.medium.com/cdk-pipelines-beyond-basics-37b731b7a182

hope you will find it useful!

[blaenk]

Sorry for the late reply @maciekbb but thank you for sharing. Your approach is clever and in my opinion the right way to go about this. I was aware of outputting the outputs to a file but it didn’t occur to me at the time. Nice job!