CLI: Using the CDK programmatically

The CDK makes it really easy to deploy infrastructure as code, but using the CDK itself is limited to the command line. We should have easy access to boostrap, diff, deploy, and destroy functionality within code.

Use Case

I’m developing a multi-tenant app with custom domains, which hits a lot of AWS quota limits, so I have to spread my user accounts across an arbitrary number of sub-accounts.

When a user signs up, I have to assign them to an account with space and deploy a customized stack with their config details. I have to retrigger deploys when those details change. I also need to update all of these stacks across all of the subaccounts when I make infrastructure updates.

Similarly, I have to monitor account usage. When a threshold is met, I have to create a new account and deploy a shared infrastructure stack for when the new users start getting assigned to it. As above, I occasionally have to update all of them at once.

It seems the best solution is to use CDK from lambdas which are either triggered on change or scheduled, depending on use case. But I find myself having to navigate around calls to the CDK, where what I really want to do is weave my new App() construction with calls like cdk.deploy(stack) or some such.

If the point of the CDK is to make Infrastructure-as-Code easy, then shouldn’t the CDK commands be as easy to program as the infrastructure itself?

Proposed Solution

I think the ideal solution would be simply make the full functionality of the CLI command available programmatically. I hesitate to suggest specifics beyond the idea that the real ideal would be to pass your stacks as arguments directly, as well being able to entirely configure the command in code, rather than environment vars and settings files.

I’ve dug a little into the codebase, and it looks like the commands are in a strange in-between space. It’s almost possible to call a few of them, but only with a lot of manual wiring (if at all). If I’m mistaken, and there’s already an easy way to do this, I’d love to hear it. I’ve yet to find a good solution googling.

Thanks!

---

This is a 🚀 Feature Request

[NGL321]

I have decided to reopen this issue due to continued frustration with this feature-gap, as seen here:

• (aws-cloudformation) Deploy a CDK stack in a Lambda · Issue #15447 · aws/aws-cdk · GitHub
• How to deploy CDK app via Lambda · Issue #2637 · aws/aws-cdk · GitHub
• Programatically expose the output of CDK list · Issue #8436 · aws/aws-cdk · GitHub
• Can we run synth programmatically? · Issue #1467 · aws/aws-cdk · GitHub

The frustration is significant enough for there to be published overly complex workarounds (ie. this, this, and this)

I believe it is harder to get traction and gague community interest/need from the RFC alone (due to only appearing in a separate repo).

[asquithea]

Adding my voice here:

For v1, I built a serverless application that provides templates that can populate a cdk.Stack. A lambda then uses CloudFormationDeployments to setup the CloudFormation changeset (i.e. without executing). Combined with the SNS notifications and a client request token, this can then execute asynchronously and we can monitor the deployment progress from another lambda.

As with some of the other requests linked above, I found there’s a bit of a gap between the obvious “call .synth()” and being able to pass control to the standard CloudFormation SDK client - and that’s only really covered by CloudFormationDeployments. An official API would be great, and the bulk of the logic is really just in that one class.

Since CloudFormation can take quite a long time, it would be great if this can continue to support generating change-sets, specifying the notification arns and passing a client request token - as these three options combine to make it fairly easy to build a solution that runs asynchronously.