API Gateway: Too Many Requests on API creation

Hello,

When creating an API that contains a lot of endpoints, we reach the API Gateway limit on resource creation and get the error

Too Many Requests (Service: ApiGateway, Status Code: 429, ...

The limits : https://docs.amazonaws.cn/en_us/batch/latest/userguide/service_limits.html

Reproduction Steps

Create a REST API with a lot of resources.

What did you expect to happen?

I expected that CDK will consider this and have a ‘sleep’ between calls if necessary.

Right now I’m just commenting some of the nested stack that contains my ressources and unccoment them in batch.

Linked to this I think : https://github.com/aws-cloudformation/cloudformation-coverage-roadmap/issues/589

What actually happened?

Got the 429 error

Environment

• CDK CLI Version : 1.109.0 (build c647e38)
• Node.js Version: v12.18.1

This is 🐛 Bug Report

[jweyrich]

Like someone already said, the problem is the rate limit of the API Gateway’s own APIs. The CreateResource is limited to 5 per second per account.

We’re facing the same problem with the Serveless Framework. Nobody solved it properly. AWS premium support suggests introducing DependsOn, but it’s not a definitive solution for sure. The 2nd link below shows AWS published a private resource type Community::CloudFormation::Delay, which also doesn’t feel like a definitive solution alone. We thought of using WaitCondition, but it’s about the same. I believe AWS should be able to handle the throttling between its service calls transparently. The “user” is not making these service-to-service calls. IMHO, since the “user” is providing a valid template that could be fully deployed, if we ignore the rate limit for APIs that the “user” itself is not calling, it should work flawlessly. However, it may become a hard optimization problem to solve.

This issue is related to:

1. https://github.com/aws-cloudformation/cloudformation-coverage-roadmap/issues/1095
2. https://github.com/aws-cloudformation/cloudformation-coverage-roadmap/issues/589

[besh0y]

Hello @peterwoodworth @nija-at I’ve been struggling with this issue for quite some time and I’d like to re-open it.

I’ve created this: repository, that consistently replicates the issue, please feel free to check it out.

The code creates a REST API with a large number of endpoints and methods, all created in multiple nested stacks. Deployment of those stacks always fails, returning a too many requests error, also rollback fails for the same reason.

I could temporarily avoid this problem by decreasing the number of resources in each nested stack and making them depend on each other during deployment so they don’t get deployed in parallel, but it’s much slower and inefficient.