CreateChangeSet occurs for every stack on every run, but change sets are never deleted
See original GitHub issueEvery Stacker run executes a Create Change Set API call on every stack to determine if changes exist for specific stack. These change sets are not deleted and there is indeed a max number of change sets allowed. The change sets are cleared only when stacks are updated, when the change set is executed. When the limit is hit, stacker is no longer able to run. The exception thrown is below:
Command executed: stacker build ###.env ./###.yaml -r us-east-1 --replacements-only -i
11:29:10 Traceback (most recent call last):
11:29:10 File "/app/jenkins/workspace/venv/bin/stacker", line 9, in <module>
11:29:10 args.run(args)
11:29:10 File "/app/jenkins/workspace/venv/lib/python2.7/site-packages/stacker/commands/stacker/build.py", line 46, in run
11:29:10 dump=options.dump)
11:29:10 File "/app/jenkins/workspace/venv/lib/python2.7/site-packages/stacker/actions/base.py", line 131, in execute
11:29:10 self.run(*args, **kwargs)
11:29:10 File "/app/jenkins/workspace/venv/lib/python2.7/site-packages/stacker/actions/build.py", line 359, in run
11:29:10 plan.execute()
11:29:10 File "/app/jenkins/workspace/venv/lib/python2.7/site-packages/stacker/plan.py", line 292, in execute
11:29:10 if not self._single_run():
11:29:10 File "/app/jenkins/workspace/venv/lib/python2.7/site-packages/stacker/plan.py", line 252, in _single_run
11:29:10 status = step.run()
11:29:10 File "/app/jenkins/workspace/venv/lib/python2.7/site-packages/stacker/plan.py", line 81, in run
11:29:10 return self._run_func(self.stack, status=self.status)
11:29:10 File "/app/jenkins/workspace/venv/lib/python2.7/site-packages/stacker/actions/build.py", line 288, in _launch_stack
11:29:10 force_interactive=stack.protected)
11:29:10 File "/app/jenkins/workspace/venv/lib/python2.7/site-packages/stacker/providers/aws/default.py", line 755, in update_stack
11:29:10 **kwargs)
11:29:10 File "/app/jenkins/workspace/venv/lib/python2.7/site-packages/stacker/providers/aws/default.py", line 775, in interactive_update_stack
11:29:10 service_role=self.service_role, **kwargs
11:29:10 File "/app/jenkins/workspace/venv/lib/python2.7/site-packages/stacker/providers/aws/default.py", line 312, in create_change_set
11:29:10 kwargs=args
11:29:10 File "/app/jenkins/workspace/venv/lib/python2.7/site-packages/stacker/providers/aws/default.py", line 81, in retry_on_throttling
11:29:10 retry_checker=_throttling_checker)
11:29:10 File "/app/jenkins/workspace/venv/lib/python2.7/site-packages/stacker/util.py", line 69, in retry_with_backoff
11:29:10 return function(*args, **kwargs)
11:29:10 File "/app/jenkins/workspace/venv/lib/python2.7/site-packages/botocore/client.py", line 312, in _api_call
11:29:10 return self._make_api_call(operation_name, kwargs)
11:29:10 File "/app/jenkins/workspace/venv/lib/python2.7/site-packages/botocore/client.py", line 605, in _make_api_call
11:29:10 raise error_class(parsed_response, operation_name)
11:29:10 botocore.errorfactory.LimitExceededException: An error occurred (LimitExceededException) when calling the CreateChangeSet operation: ChangeSet limit exceeded for stack arn:aws:cloudformation:us-east-1:141627368605:stack/#############/14dc3c00-602a-11e7-a567-500c286e1a36
11:29:10 NOTE: FAILURE DETECTED - last stacker run failed, sleeping 10 seconds to back off API and marking job to fail...
Issue Analytics
- State:
- Created 6 years ago
- Reactions:1
- Comments:5 (5 by maintainers)
Top Results From Across the Web
Updating stacks using change sets - AWS CloudFormation
View the change set to see which stack settings and resources will change. For example, you can see which resources CloudFormation will add,...
Read more >Understanding change sets - AWS CloudFormation Workshop
In the CloudFormation console, select the changesets-workshop stack, and from Stack actions, choose Create change set for current stack. From Prepare template, ...
Read more >CloudFormation — Boto3 Docs 1.26.36 documentation - AWS
CloudFormation creates and deletes all member resources of the stack ... You can't modify your stack set's execution configuration while there are running...
Read more >ValidationError Stack:arn aws cloudformation stack is in ...
This happens when stack creation fails. By default the stack will remain in place with a status of ROLLBACK_COMPLETE .
Read more >CloudFormation Stack & Change Set Best Practices - Reddit
Make those changes in the CF template. Do a dry run of the changes to see what would happen, via create-change-set.
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
changeset_nuke.py.zip
@phobologic and @russellballestrini - this is our hack job to fix the glitch on previously created change sets. It’s hacky, but works.
I just ran into this. When you run stacker in interactive mode, it will always create a change set, and if there’s no changes the change set will be “FAILED” and never removed by stacker. The interactive provider should just delete the changeset if there’s no changes.