(pipelines): Allow re-generating CloudFormation changeset after failure

As of today, I couldn’t find a way to re-run an application stage deployment if application of the CloudFormation changeset failed. We can kick off the pipeline again by committing a dummy/no-op commit to the repository, but this creates unnecessary cruft in the SCM history.

Use Case

Consider a CloudFormation template that could not apply for some reason. In my case, it was because of a naming conflict with another CloudFormation stack that needed to be destroyed.

The CodePipeline user interface looks like this after the failure:

So, I took a look at the failure and noticed the naming conflict. I went and destroyed the other CDK stack and wanted to re-run the deployment. However, when I clicked the “Retry” button in the UI, the “Deploy” stage failed with this error message:

ChangeSet [PipelineChange] does not exist (Service: AmazonCloudFormation; Status Code: 404; Error Code: ChangeSetNotFound; Request ID: <redacted>)

Indeed, in the CloudFormation user interface, the changeset was not present:

The problem appears to be because there’s no way to retry an entire stage in CodePipeline, including steps that previously succeeded. According to the aws cli docs (aws codepipeline retry-stage-execution help):

--retry-mode (string)
  The scope of the retry attempt. Currently, the only supported  value
  is FAILED_ACTIONS.

  Possible values:

  o FAILED_ACTIONS

Because I didn’t have a way to rerun the .Prepare step (which produces the changeset), I was kind of stuck and had to revert to pushing a no-op commit to force the step to re-run.

Proposed Solution

Since this appears to be a CodePipeline limitation, maybe a custom “step” is needed? Like maybe an SNS topic or something to kick off the deployment again? Or maybe a better solution would be to +1 feature requests on the CodePipeline side to allow “Retry (including succeeded)” behavior?

Other

• 👋 I may be able to implement this feature request
• ⚠️ This feature might incur a breaking change

---

This is a 🚀 Feature Request

[openwebsolns]

This is still a problem. If you have a large pipeline, “releasing a change” would require deploying through all the previous stages before you can retry the failed stage. Is there a creative workaround by which we can “replay” a changeset with the same name (PipelineChange) so that the CodePipeline retry works? Does CDK provide the ability to just create the changeset?

[fab-mindflow]

Indeed, we also have a large CDK deployment and this forces us to build everything again and again. Any workaround?