Garbage-collect orphaned stacks

❓ General Issue

The Question

cdk deploy leaves stack deployed if a stack is removed from application post deployment and cdk app is deployed again.

Steps to Reproduce

1. Create a CDK app with two stacks
2. Run: cdk deploy --all
3. Delete one stack from the CDK app
4. Run: cdk deploy --all

Expected Results The deleted stack is gone.

Actual Results The deleted stack exists and has been “leaked”.

i.e. the stacks are procedural rather than declarative. This doesn’t fit into a managed pipeline model as stacks will accidentally leak and have to be manually cleaned, while the behavior makes sense if you’re familiar with the implementation details of CDK, this isn’t the right developer experience.

Environment

• CDK CLI Version: 1.73.0
• Module Version:
• Node.js Version: 12.x and 15.2.0
• OS: macOS Mojave and CodeBuild Ubuntu (not used other OSes)
• Language (Version): TypeScript (4.0.7)

Other information

[tomas-mazak]

Hi, I would like to revive this topic. In our company, we use CDK extensively and especially with CDK pipelines (we have 120+ pipelines ATM) the orphan stacks are becoming an issue.

The advantage of CDK pipeline is that once you instantiate it, you can forget about applying CDK changes - you simply push to git and changes are applied by a pipeline. However, if you add a pipeline “stage” and later remove it (or just rename it), the stage stacks will stay up and will be “orphaned”.

We tackle this by assigning a set of tags to each stack that uniquely identify the CDK app. We provide a tool to application owners that synthesizes the app and gets the list of “declared” stacks, then retrieves the “running” stacks - stacks with corresponding tags from CFN API - and deleting “running” that are not “declared”.

However, this tool needs to be run manually by application owners. Also, I believe many people must be having the same issue. Would it make sense to resolve this within CDK?

The AppRegistry idea looks interesting. Potentially, simpler approach with tagging and UUIDs would work too:

• on first deploy, CDK app is assigned an auto-generated “app id” - UUID for example
• the app id is then stored in CDK context for persistence
• all stacks deployed by the app are tagged with for example aws-cdk:app-id=<app-id>

Then, a new CLI can be provided. I can see many options to implement this, for example cdk apply that would deploy new/changed stacks and delete stacks that are no longer “declared”. Or simply cdk deploy --clear-orphans. or cdk destroy --orphans-only…

@eladb what do you think? If there’s an interest, I would be happy to look into this

[zhelyan]

The only way I could think this can be solved is by keeping some inventory in the environment of all CDK stacks

An App is a container for one or more stacks: it serves as each stack’s scope

Can we track the App state in a dedicated “app” stack?

Edit: Sorry, I am not being clear. Make App deploy an “app” stack which then uses e.g. https://github.com/aws/aws-cdk/issues/13676#issuecomment-809674456 to track the state of the stacks included in the app. In future, the dedicated app stack can also wrap any app management features e.g. creating a resource group for the app, running periodic drift detection on the app stacks etc.