(@aws-cdk/aws-apigateway): Wrong Logical ID for UsagePlanKeyResource

❓ General Issue

The Question

Hi there! We migrated a lot of APIs from our on-premise-solution into the AWS API Gateway. To control access to our APIs we configured Cognito (with OAuth Scopes) and API Keys. These API Keys are generated with an array holding a number of subscribers (other stakeholders or applications):

subscriber: ['user1', 'user2', 'user3']

Our code iterates over this array, generates API Keys for each element and adds them to the APIs UsagePlan:

export const addApiKeysToUsagePlan = (api: RestApi, name: string, apiSubscriber: string[], usagePlan: UsagePlan): void => {
  apiSubscriber.forEach(function (subscriber: string) {
    const apiKey = new ApiKey(api, name + '-apiKey-' + subscriber, {
      apiKeyName: name + '-' + subscriber,
    });
    usagePlan.addApiKey(apiKey);
  });
};

This is the output of a “cdk diff” when adding three freshly new subscriber to a already deployed API:

[-] AWS::ApiGateway::Deployment testapiDeployment356D2C350f0d9c33af21b1b5acbe01dd96f30695 destroy
[+] AWS::ApiGateway::Deployment test-api/Deployment testapiDeployment356D2C35b1f6494fcf179c88cb95f6e66e796231
[+] AWS::ApiGateway::UsagePlanKey test-api/test-api/UsagePlanKeyResource testapiUsagePlanKeyResource6D2F267F
[+] AWS::ApiGateway::UsagePlanKey test-api/test-api/UsagePlanKeyResource:testapistacktestapitestapiapiKeyuser2A184AC36 testapiUsagePlanKeyResourcetestapistacktestapitestapiapiKeyuser2A184AC36802022E8
[+] AWS::ApiGateway::UsagePlanKey test-api/test-api/UsagePlanKeyResource:testapistacktestapitestapiapiKeyuser315D7663C testapiUsagePlanKeyResourcetestapistacktestapitestapiapiKeyuser315D7663CF36A0768
[+] AWS::ApiGateway::ApiKey test-api/test-api-apiKey-user1 testapitestapiapiKeyuser100D43FD3
[+] AWS::ApiGateway::ApiKey test-api/test-api-apiKey-user2 testapitestapiapiKeyuser26097BDA4
[+] AWS::ApiGateway::ApiKey test-api/test-api-apiKey-user3 testapitestapiapiKeyuser372A97416
[~] AWS::ApiGateway::Stage test-api/DeploymentStage.apigw testapiDeploymentStageapigwEA4DAFFA
 └─ [~] DeploymentId
     └─ [~] .Ref:
         ├─ [-] testapiDeployment356D2C350f0d9c33af21b1b5acbe01dd96f30695
         └─ [+] testapiDeployment356D2C35b1f6494fcf179c88cb95f6e66e796231
[~] AWS::ApiGateway::Method test-api/Default/GET testapiGETD8DE4ED1
 └─ [~] AuthorizationScopes
     └─ @@ -1,1 +1,5 @@
        [-] []
        [+] [
        [+]   "subscriber/user1",
        [+]   "subscriber/user2",
        [+]   "subscriber/user3"
        [+] ]

You can see, that three new API Keys will be added to the UsagePlan. Please note the differences here:

[+] AWS::ApiGateway::UsagePlanKey test-api/test-api/UsagePlanKeyResource testapiUsagePlanKeyResource6D2F267F
[+] AWS::ApiGateway::UsagePlanKey test-api/test-api/UsagePlanKeyResource:testapistacktestapitestapiapiKeyuser2A184AC36 testapiUsagePlanKeyResourcetestapistacktestapitestapiapiKeyuser2A184AC36802022E8
[+] AWS::ApiGateway::UsagePlanKey test-api/test-api/UsagePlanKeyResource:testapistacktestapitestapiapiKeyuser315D7663C testapiUsagePlanKeyResourcetestapistacktestapitestapiapiKeyuser315D7663CF36A0768

The Logical ID of the first UsagePlanKeyResource differs heavily from the other two. This is a screenshot from the AWS console to make things more clear (the screenshot was taken earlier, thats why the IDs of the cdk diff and the aws console don’t match - please understand the screenshot as a visual support):

There are three API Keys and they are all related to the UsagePlan with a UsagePlanKey, with the exception of the first UsagePlanKeys Logical ID being different.

With this issue in mind I could remove user2 and/or user3 from the array with no problems (cdk diff):

[-] AWS::ApiGateway::Deployment testapiDeployment356D2C35b1f6494fcf179c88cb95f6e66e796231 destroy
[-] AWS::ApiGateway::UsagePlanKey testapiUsagePlanKeyResourcetestapistacktestapitestapiapiKeyuser2A184AC36802022E8 destroy
[-] AWS::ApiGateway::UsagePlanKey testapiUsagePlanKeyResourcetestapistacktestapitestapiapiKeyuser315D7663CF36A0768 destroy
[-] AWS::ApiGateway::ApiKey testapitestapiapiKeyuser26097BDA4 destroy
[-] AWS::ApiGateway::ApiKey testapitestapiapiKeyuser372A97416 destroy
[+] AWS::ApiGateway::Deployment test-api/Deployment testapiDeployment356D2C35f7dc1c595d8e9063183b9e1c925537c0
[~] AWS::ApiGateway::Stage test-api/DeploymentStage.apigw testapiDeploymentStageapigwEA4DAFFA
 └─ [~] DeploymentId
     └─ [~] .Ref:
         ├─ [-] testapiDeployment356D2C35b1f6494fcf179c88cb95f6e66e796231
         └─ [+] testapiDeployment356D2C35f7dc1c595d8e9063183b9e1c925537c0
[~] AWS::ApiGateway::Method test-api/Default/GET testapiGETD8DE4ED1
 └─ [~] AuthorizationScopes
     └─ @@ -1,5 +1,3 @@
        [ ] [
        [-]   "subscriber/user1",
        [-]   "subscriber/user2",
        [-]   "subscriber/user3"
        [+]   "subscriber/user1"
        [ ] ]

But when I try to remove user1 instead, the stack fails because of a replacement:

[-] AWS::ApiGateway::Deployment testapiDeployment356D2C35b1f6494fcf179c88cb95f6e66e796231 destroy
[-] AWS::ApiGateway::UsagePlanKey testapiUsagePlanKeyResourcetestapistacktestapitestapiapiKeyuser2A184AC36802022E8 destroy
[-] AWS::ApiGateway::ApiKey testapitestapiapiKeyuser100D43FD3 destroy
[+] AWS::ApiGateway::Deployment test-api/Deployment testapiDeployment356D2C35d4202d75979c652d5556c72542102bf6
[~] AWS::ApiGateway::Stage test-api/DeploymentStage.apigw testapiDeploymentStageapigwEA4DAFFA
 └─ [~] DeploymentId
     └─ [~] .Ref:
         ├─ [-] testapiDeployment356D2C35b1f6494fcf179c88cb95f6e66e796231
         └─ [+] testapiDeployment356D2C35d4202d75979c652d5556c72542102bf6
[~] AWS::ApiGateway::Method test-api/Default/GET testapiGETD8DE4ED1
 └─ [~] AuthorizationScopes
     └─ @@ -1,5 +1,4 @@
        [ ] [
        [-]   "subscriber/user1",
        [ ]   "subscriber/user2",
        [ ]   "subscriber/user3"
        [ ] ]
[~] AWS::ApiGateway::UsagePlanKey test-api/test-api/UsagePlanKeyResource testapiUsagePlanKeyResource6D2F267F replace
 └─ [~] KeyId (requires replacement)
     └─ [~] .Ref:
         ├─ [-] testapitestapiapiKeyuser100D43FD3
         └─ [+] testapitestapiapiKeyuser26097BDA4

3/8 | 15:21:45 | UPDATE_IN_PROGRESS | AWS::ApiGateway::UsagePlanKey | test-api/test-api/UsagePlanKeyResource (testapiUsagePlanKeyResource6D2F267F) Requested update requires the creation of a new physical resource; hence creating one. 3/8 | 15:21:45 | UPDATE_FAILED | AWS::ApiGateway::UsagePlanKey | test-api/test-api/UsagePlanKeyResource (testapiUsagePlanKeyResource6D2F267F) ezplnhe139:vxxai7 already exists in stack arn:aws:cloudformation:eu-central-1:318255054997:stack/test-api-stack/63771480-2e4d-11eb-83d4-02886036c898

I can’t figure out why this happens. There is no difference in the code regarding the array elements 0,1,…,n. Please note: When adding just one API Key (or just one array element) I can remove it without problems, but the Logical IDs strange look remains.

The replacement of the KeyId happens only if the number of API Keys is greater than one and the to be removed element is the first one. Is this some kind of bug in the cdk?

Any help is greatly appreciated - thanks!

Kind Regards, Marcus

Environment

• CDK CLI Version: 1.76.0 (build c207717)
• Module Version: 1.76.0
• Node.js Version: v12.19.0
• OS: Windows 10
• Language (Version): TypeScript (4.0.5) | Python (3.9.0)

[ghost]

Hi @nija-at!

I had to make the following changes to apply the fix to our already deployed stacks:

• Renamed the ID of the UsagePlan, otherwise the stack fails because of a duplicate UsagePlanKey (create before destroy)
• Removed the DeploymentStage, otherwise the stack fails because it tries to apply a single API Key to multiple Stages (‘Usage Plan xxx cannot be added because API Key yyy cannot reference multiple Usage Plans with the same API Stage’)

After the deployment with these changes and the upgrade to 1.87.0 I have reverted these changes and deployed the stacks again, so the UsagePlan IDs and Stages were as before.

This is our original code without the changes:

export const addUsagePlan = (api: RestApi, name: string, rateLimit: number, burstLimit: number): UsagePlan => {
  const usagePlan = api.addUsagePlan(name, {
    name: name,
    throttle: {
      rateLimit,
      burstLimit,
    },
  });

  usagePlan.addApiStage({
    stage: api.deploymentStage,
  });

  return usagePlan;
};

And now with the mentioned changes:

export const addUsagePlan = (api: RestApi, name: string, rateLimit: number, burstLimit: number): UsagePlan => {
  const usagePlan = api.addUsagePlan(name + '-new', {
    name: name,
    throttle: {
      rateLimit,
      burstLimit,
    },
  });

  /*usagePlan.addApiStage({
    stage: api.deploymentStage,
  });*/

  return usagePlan;
};

Maybe these information are useful to you.

[yosefbs]

@brad-lucas try to add :

"context": {
    "@aws-cdk/aws-apigateway:usagePlanKeyOrderInsensitiveId": false
  }

in your cdk.json file. in cdk1 default value(for existing project) was false but when updated to cdk2 the default value is true