(aws-ecs): can't update ECS cluster because of "capacity provider is in use"

A previously (before 1.108.0) created ECS cluster with FARGATE and FARGATE_SPOT capacity providers can’t be updated anymore because of this error:

Error occurred during operation 'putClusterCapacityProviders SDK error: The specified capacity provider is in use and cannot be removed.

Reproduction Steps

Create an ECS cluster with

capacityProviders: ["FARGATE", "FARGATE_SPOT"],

with a CDK version prior to 1.108.0

Then upgrade CDK to latest version (1.110.1), try to update the stack to

enableFargateCapacityProviders: true,

(because the old property is deprecated)

What did you expect to happen?

Capacity providers should be set correctly and cluster should be updated

What actually happened?

Resource handler returned message: “Error occurred during operation ‘putClusterCapacityProviders SDK error: The specified capacity provider is in use and cannot be removed. (Service: AmazonECS; Status Code: 400; Error Code: ResourceInUseException; Request ID: ee1d2754-517d-4148-bbb1-25d0f8a1aa88; Proxy: null)’.” (RequestToken: 3d87b953-a468-bffc-d442-862a457b1f9b, HandlerErrorCode: GeneralServiceException)

Environment

• CDK CLI Version : 1.110.1
• Framework Version: 1.110.1
• Node.js Version: v14.17.0
• OS : MacOS 11.4
• Language (Version): Typescript 4.1.2

Other

Probably caused by https://github.com/aws/aws-cdk/commit/6b2d0e0c867651cd632be9ca99c6e342fb3c1067

---

This is 🐛 Bug Report

[madeline-k]

The issue was introduced with this bug fix: https://github.com/aws/aws-cdk/pull/15012/

Unfortunately, we are now between a rock and a hard place, because if we just revert that change, customers who have deployed with 1.108.0 and up could run into this same issue.

In the meantime you can apply this workaround to your cluster. This is the best I could come up with, there might be a simpler way.

import * as cdk from '@aws-cdk/core';
import * as ecs from '@aws-cdk/aws-ecs';

export class MyStack extends cdk.Stack {
  constructor(scope: cdk.Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    const cluster = new ecs.Cluster(this, 'MyCluster', {
      enableFargateCapacityProviders: true,
    });

    // Add back the old method of specifying capacity providers
    const cfnCluster = cluster.node.defaultChild as ecs.CfnCluster;
    cfnCluster.capacityProviders = ['FARGATE', 'FARGATE_SPOT'];

    // Remove the new method of specifying capacity providers. 
    cdk.Aspects.of(this).add(new MyAspect());
  }
}

class MyAspect implements cdk.IAspect {
  public visit(node: cdk.IConstruct): void {
    if (node instanceof ecs.CfnClusterCapacityProviderAssociations) {
      // IMPORTANT: The id supplied here must be the same as the id of your cluster. Don't worry, you won't remove the cluster.  
      node.node.scope?.node.tryRemoveChild('MyCluster');
    }
  }
}

Note that a simple cluster.node.tryRemoveChild('MyCluster') doesn’t work here, which is why I used an Aspect to remove the CfnClusterCapacityProviderAssociations resource. I am not sure exactly why this is. I think it could be because the resource was added with an Aspect here.

[madeline-k]

Looks like this is a breaking change and we should try to fix this as soon as possible. I am going to try to repro it.