Dependency stacks cause update failure

Note: for support questions, please first reference our documentation, then use Stackoverflow. This repository’s issues are intended for feature requests and bug reports.

• I’m submitting a …

  • 🪲 bug report
  • 🚀 feature request
  • 📚 construct library gap
  • ☎️ security issue or vulnerability => Please see policy
  • ❓ support request => Please see note at the top of this template.

• What is the current behavior? If the current behavior is a 🪲bug🪲: Please provide the steps to reproduce

stack_1 = FirstStack(
    app=app,
    id='FirstStack
)

stack_2 = SecondStack(
    app=app,
    id='SecondStack',
    construct_from_stack_1=stack1.some_construct
)

This causes a dependency via stack output. When I decide not to use construct_from_stack_1 anymore (by deleting its usage from stack_2), the stack_2 fails to update - for instance:

eks-dev
eks-dev: deploying...
eks-dev: creating CloudFormation changeset...
 0/1 | 12:13:45 | UPDATE_ROLLBACK_IN_P | AWS::CloudFormation::Stack              | eks-dev Export eks-dev:ExportsOutputFnGetAttEksElasticLoadBalancer4FCBC5E7SourceSecurityGroupOwnerAlias211654CC cannot be deleted as it is in use by ports-assignment-dev

 ❌  eks-dev failed: Error: The stack named eks-dev is in a failed state: UPDATE_ROLLBACK_COMPLETE
The stack named eks-dev is in a failed state: UPDATE_ROLLBACK_COMPLETE

Looks like CDK tries to delete resource in wrong order - starting from the output first rather than its usage in dependent stacks and then from the souce stack itself.

• What is the expected behavior (or behavior of feature suggested)? Update removes resources that are no longer used

• What is the motivation / use case for changing the behavior or adding this feature?

Life-time dependecies are created which prevents dependent stacks from being updated.

• Please tell us about your environment:

  • CDK CLI Version: 1.0.0
  • Module Version: 1.0.0
  • OS: [all]
  • Language: [all ]

• Other information (e.g. detailed explanation, stacktraces, related issues, suggestions how to fix, links for us to have context, eg. associated pull-request, stackoverflow, gitter, etc)

[nakedible]

This bug is caused by the automatic dependency resolution mechanism in the CDK CLI, which means that when you update stack_2, it will automatically update stack_1 first (which obviously fails as stack_2 is still using the exported resource). The solution is really simple - just say cdk deploy -e stack_2, which will update only stack_2, and afterwards you can say cdk deploy stack_1 to clean up the unused export.

This will fail if you at the same time add something to stack_1 that is needed by stack_2 - in this case, stack_2 cannot be updated first, but neither can stack_1 because of the export. This is an obvious limitation with CloudFormation that has nothing to do with CDK, and the simplest way to avoid this is to just to do smaller changes.

The proper way to solve all problems like this is to use NestedStack instead of Stack. Automated support for that landed in 1.12.0, and that allows CloudFormation to handle this case correctly - first by creating all new resources in all stacks in dependency order, then updating all references and then only finally doing a pass to remove all the replaced resources.

Not sure what should actually be done about this in CDK - one solution would be to just add a note when a stack update fails to an export being used that “perhaps try updating stack with --exclusively”.

[kichik]

As @nakedible said, one of the workarounds is splitting the deploy into two steps. The -e flag must be used so CDK doesn’t deploy all stacks. Here is an example of that.

# first step will remove the usage of the export
cdk deploy --exclusively SecondStack
# second step can now remove the export
cdk deploy --all