Lightrun
question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Fails on too many simultaneous API calls

See original GitHub issue

This may be a config issue with my Kong setup, but I have quite a lot of upstreams, and kongfig basically calls it so fast (with no delay) that it triggers rate limits / connection limits.

Can we put it into a mode where it calls serially and only ever make a single connection to Kong at a time? The speed difference I’m sure would be marginal.

Errors:

2018/01/30 16:02:10 [error] 25#0: *280 [lua] responses.lua:107: handler(): FATAL: sorry, too many clients already, client: 127.0.0.1, server: kong_admin, request: "GET /upstreams/2b1fbadb-02b8-4538-b97e-5afde1f22063/targets/active/ HTTP/1.1", host: "127.0.0.1:8001"
2018/01/30 16:02:10 [error] 25#0: *284 [lua] responses.lua:107: handler(): FATAL: sorry, too many clients already, client: 127.0.0.1, server: kong_admin, request: "GET /upstreams/2ffae228-ff19-4368-9d2c-16a26231197e/targets/active/ HTTP/1.1", host: "127.0.0.1:8001"
2018/01/30 16:02:10 [error] 25#0: *252 [lua] responses.lua:107: handler(): FATAL: remaining connection slots are reserved for non-replication superuser connections, client: 127.0.0.1, server: kong_admin, request: "GET /upstreams/33862f0c-5162-4471-b064-a11d3b239e77/targets/active/ HTTP/1.1", host: "127.0.0.1:8001"
2018/01/30 16:02:10 [error] 25#0: *291 [lua] responses.lua:107: handler(): FATAL: sorry, too many clients already, client: 127.0.0.1, server: kong_admin, request: "GET /upstreams/5a74b309-d639-4a94-9cfb-8881f5603890/targets/active/ HTTP/1.1", host: "127.0.0.1:8001"
2018/01/30 16:02:10 [error] 25#0: *263 [lua] responses.lua:107: handler(): FATAL: remaining connection slots are reserved for non-replication superuser connections, client: 127.0.0.1, server: kong_admin, request: "GET /upstreams/24e3929e-a221-443b-8a39-c8a4d0bb861c/targets/active/ HTTP/1.1", host: "127.0.0.1:8001"
2018/01/30 16:02:10 [error] 25#0: *253 [lua] responses.lua:107: handler(): FATAL: remaining connection slots are reserved for non-replication superuser connections, client: 127.0.0.1, server: kong_admin, request: "GET /upstreams/088ab582-5d75-4bd8-b5ef-5e51f6ff37a5/targets/active/ HTTP/1.1", host: "127.0.0.1:8001"
2018/01/30 16:02:10 [error] 25#0: *266 [lua] responses.lua:107: handler(): FATAL: remaining connection slots are reserved for non-replication superuser connections, client: 127.0.0.1, server: kong_admin, request: "GET /upstreams/9db80255-2b47-4d00-9953-c9b9cdbade68/targets/active/ HTTP/1.1", host: "127.0.0.1:8001"
2018/01/30 16:02:10 [error] 25#0: *265 [lua] responses.lua:107: handler(): FATAL: remaining connection slots are reserved for non-replication superuser connections, client: 127.0.0.1, server: kong_admin, request: "GET /upstreams/19e8102b-003b-4f02-9ed1-e661c9f49ea0/targets/active/ HTTP/1.1", host: "127.0.0.1:8001"
2018/01/30 16:02:10 [error] 25#0: *278 [lua] responses.lua:107: handler(): FATAL: remaining connection slots are reserved for non-replication superuser connections, client: 127.0.0.1, server: kong_admin, request: "GET /upstreams/89265874-e443-44c1-8df2-a025f8486be8/targets/active/ HTTP/1.1", host: "127.0.0.1:8001"

On the following sort of access log:

127.0.0.1 - - [30/Jan/2018:16:02:10 +0000] "GET /upstreams HTTP/1.1" 200 7833 "-" "node-fetch/1.0 (+https://github.com/bitinn/node-fetch)"
127.0.0.1 - - [30/Jan/2018:16:02:10 +0000] "GET /upstreams/e7a6757f-fbcc-441b-9096-0e8fc7aeb972/targets/active/ HTTP/1.1" 200 185 "-" "node-fetch/1.0 (+https://github.com/bitinn/node-fetch)"
127.0.0.1 - - [30/Jan/2018:16:02:10 +0000] "GET /upstreams/22964306-e9e1-4857-b844-7bb7d3460fdb/targets/active/ HTTP/1.1" 200 186 "-" "node-fetch/1.0 (+https://github.com/bitinn/node-fetch)"
127.0.0.1 - - [30/Jan/2018:16:02:10 +0000] "GET /upstreams/3d8d45b9-1cc4-4c6b-8c05-81c9b149f09e/targets/active/ HTTP/1.1" 200 185 "-" "node-fetch/1.0 (+https://github.com/bitinn/node-fetch)"
127.0.0.1 - - [30/Jan/2018:16:02:10 +0000] "GET /upstreams/f4f0d25e-34f6-4463-820b-bf5c58e8acec/targets/active/ HTTP/1.1" 200 186 "-" "node-fetch/1.0 (+https://github.com/bitinn/node-fetch)"
127.0.0.1 - - [30/Jan/2018:16:02:10 +0000] "GET /upstreams/2b1fbadb-02b8-4538-b97e-5afde1f22063/targets/active/ HTTP/1.1" 500 54 "-" "node-fetch/1.0 (+https://github.com/bitinn/node-fetch)"
127.0.0.1 - - [30/Jan/2018:16:02:10 +0000] "GET /upstreams/2ffae228-ff19-4368-9d2c-16a26231197e/targets/active/ HTTP/1.1" 500 54 "-" "node-fetch/1.0 (+https://github.com/bitinn/node-fetch)"
127.0.0.1 - - [30/Jan/2018:16:02:10 +0000] "GET /upstreams/33862f0c-5162-4471-b064-a11d3b239e77/targets/active/ HTTP/1.1" 500 54 "-" "node-fetch/1.0 (+https://github.com/bitinn/node-fetch)"
127.0.0.1 - - [30/Jan/2018:16:02:10 +0000] "GET /upstreams/5a74b309-d639-4a94-9cfb-8881f5603890/targets/active/ HTTP/1.1" 500 54 "-" "node-fetch/1.0 (+https://github.com/bitinn/node-fetch)"
127.0.0.1 - - [30/Jan/2018:16:02:10 +0000] "GET /upstreams/24e3929e-a221-443b-8a39-c8a4d0bb861c/targets/active/ HTTP/1.1" 500 54 "-" "node-fetch/1.0 (+https://github.com/bitinn/node-fetch)"
127.0.0.1 - - [30/Jan/2018:16:02:10 +0000] "GET /upstreams/088ab582-5d75-4bd8-b5ef-5e51f6ff37a5/targets/active/ HTTP/1.1" 500 54 "-" "node-fetch/1.0 (+https://github.com/bitinn/node-fetch)"
127.0.0.1 - - [30/Jan/2018:16:02:10 +0000] "GET /upstreams/9db80255-2b47-4d00-9953-c9b9cdbade68/targets/active/ HTTP/1.1" 500 54 "-" "node-fetch/1.0 (+https://github.com/bitinn/node-fetch)"
127.0.0.1 - - [30/Jan/2018:16:02:10 +0000] "GET /upstreams/19e8102b-003b-4f02-9ed1-e661c9f49ea0/targets/active/ HTTP/1.1" 500 54 "-" "node-fetch/1.0 (+https://github.com/bitinn/node-fetch)"
127.0.0.1 - - [30/Jan/2018:16:02:10 +0000] "GET /upstreams/89265874-e443-44c1-8df2-a025f8486be8/targets/active/ HTTP/1.1" 500 54 "-" "node-fetch/1.0 (+https://github.com/bitinn/node-fetch)"
127.0.0.1 - - [30/Jan/2018:16:02:10 +0000] "GET /upstreams/e8a40693-4e8f-46be-a8ff-7265dc1b9c0f/targets/active/ HTTP/1.1" 200 186 "-" "node-fetch/1.0 (+https://github.com/bitinn/node-fetch)"
127.0.0.1 - - [30/Jan/2018:16:02:10 +0000] "GET /upstreams/deef001a-838b-4b62-8d67-d77c9c42f8c4/targets/active/ HTTP/1.1" 200 186 "-" "node-fetch/1.0 (+https://github.com/bitinn/node-fetch)"
127.0.0.1 - - [30/Jan/2018:16:02:10 +0000] "GET /upstreams/1fdcbc36-c2d5-4f6e-8269-b3b12a914549/targets/active/ HTTP/1.1" 200 186 "-" "node-fetch/1.0 (+https://github.com/bitinn/node-fetch)"
127.0.0.1 - - [30/Jan/2018:16:02:10 +0000] "GET /upstreams/154fd8c9-fd3a-4030-be71-a2ec5009e1af/targets/active/ HTTP/1.1" 200 186 "-" "node-fetch/1.0 (+https://github.com/bitinn/node-fetch)"
127.0.0.1 - - [30/Jan/2018:16:02:10 +0000] "GET /upstreams/ba3b2733-47a0-4d8c-ac9e-0ce844347c9e/targets/active/ HTTP/1.1" 200 184 "-" "node-fetch/1.0 (+https://github.com/bitinn/node-fetch)"
127.0.0.1 - - [30/Jan/2018:16:02:10 +0000] "GET /upstreams/c0d09d68-eb22-4708-a24d-13784ea30555/targets/active/ HTTP/1.1" 200 186 "-" "node-fetch/1.0 (+https://github.com/bitinn/node-fetch)"
127.0.0.1 - - [30/Jan/2018:16:02:10 +0000] "GET /upstreams/0105be76-e820-471b-b047-8c7a3fb4ae2b/targets/active/ HTTP/1.1" 200 186 "-" "node-fetch/1.0 (+https://github.com/bitinn/node-fetch)"
127.0.0.1 - - [30/Jan/2018:16:02:10 +0000] "GET /upstreams/ffd07394-8774-474e-b324-adf925983a90/targets/active/ HTTP/1.1" 200 186 "-" "node-fetch/1.0 (+https://github.com/bitinn/node-fetch)"
127.0.0.1 - - [30/Jan/2018:16:02:10 +0000] "GET /upstreams/51580f91-cff1-40a5-9d7f-0f07927d1f48/targets/active/ HTTP/1.1" 200 184 "-" "node-fetch/1.0 (+https://github.com/bitinn/node-fetch)"
127.0.0.1 - - [30/Jan/2018:16:02:10 +0000] "GET /upstreams/ffc683a5-5b57-41f3-bf51-1dd93148832a/targets/active/ HTTP/1.1" 200 185 "-" "node-fetch/1.0 (+https://github.com/bitinn/node-fetch)"
127.0.0.1 - - [30/Jan/2018:16:02:10 +0000] "GET /upstreams/92e0e051-4987-465d-8c9d-4668aed90815/targets/active/ HTTP/1.1" 200 185 "-" "node-fetch/1.0 (+https://github.com/bitinn/node-fetch)"
127.0.0.1 - - [30/Jan/2018:16:02:10 +0000] "GET /upstreams/2b9784cc-a722-4709-b75c-6fa914fcb0e8/targets/active/ HTTP/1.1" 200 186 "-" "node-fetch/1.0 (+https://github.com/bitinn/node-fetch)"
127.0.0.1 - - [30/Jan/2018:16:02:10 +0000] "GET /upstreams/c53b0f34-205c-47d3-a026-70b2750aad67/targets/active/ HTTP/1.1" 200 185 "-" "node-fetch/1.0 (+https://github.com/bitinn/node-fetch)"
127.0.0.1 - - [30/Jan/2018:16:02:10 +0000] "GET /upstreams/66fa6ce9-b6b2-43a0-9006-6a88eca13830/targets/active/ HTTP/1.1" 200 185 "-" "node-fetch/1.0 (+https://github.com/bitinn/node-fetch)"
127.0.0.1 - - [30/Jan/2018:16:02:10 +0000] "GET /upstreams/d9114aca-413b-4740-85a4-49bd5f139f81/targets/active/ HTTP/1.1" 200 185 "-" "node-fetch/1.0 (+https://github.com/bitinn/node-fetch)"
127.0.0.1 - - [30/Jan/2018:16:02:10 +0000] "GET /upstreams/c44fe941-4395-4e58-a3f7-ed73e37bb470/targets/active/ HTTP/1.1" 200 185 "-" "node-fetch/1.0 (+https://github.com/bitinn/node-fetch)"
127.0.0.1 - - [30/Jan/2018:16:02:10 +0000] "GET /upstreams/7a8a692d-2866-48fe-88b5-206c84b68d2b/targets/active/ HTTP/1.1" 200 185 "-" "node-fetch/1.0 (+https://github.com/bitinn/node-fetch)"
127.0.0.1 - - [30/Jan/2018:16:02:10 +0000] "GET /upstreams/379bf778-dc6a-4824-a2ce-553385fa5fe7/targets/active/ HTTP/1.1" 200 185 "-" "node-fetch/1.0 (+https://github.com/bitinn/node-fetch)"
127.0.0.1 - - [30/Jan/2018:16:02:10 +0000] "GET /upstreams/1797ded0-6613-48bd-a844-8c338127b30b/targets/active/ HTTP/1.1" 200 185 "-" "node-fetch/1.0 (+https://github.com/bitinn/node-fetch)"
127.0.0.1 - - [30/Jan/2018:16:02:10 +0000] "GET /upstreams/d0176278-f1bc-46c9-b7ef-7ec896715866/targets/active/ HTTP/1.1" 200 185 "-" "node-fetch/1.0 (+https://github.com/bitinn/node-fetch)"
127.0.0.1 - - [30/Jan/2018:16:02:10 +0000] "GET /upstreams/3357b190-4a7e-400d-9bcc-5bb202d4aee3/targets/active/ HTTP/1.1" 200 184 "-" "node-fetch/1.0 (+https://github.com/bitinn/node-fetch)"
127.0.0.1 - - [30/Jan/2018:16:02:10 +0000] "GET /upstreams/0b1185fc-7079-4214-8e2d-ab19b43e9576/targets/active/ HTTP/1.1" 200 185 "-" "node-fetch/1.0 (+https://github.com/bitinn/node-fetch)"
127.0.0.1 - - [30/Jan/2018:16:02:10 +0000] "GET /upstreams/6b754acb-e059-499b-a1c1-6cbf7e210613/targets/active/ HTTP/1.1" 200 185 "-" "node-fetch/1.0 (+https://github.com/bitinn/node-fetch)"
127.0.0.1 - - [30/Jan/2018:16:02:10 +0000] "GET /upstreams/bf4abc87-5002-4259-8a1e-88f475557abd/targets/active/ HTTP/1.1" 200 185 "-" "node-fetch/1.0 (+https://github.com/bitinn/node-fetch)"
127.0.0.1 - - [30/Jan/2018:16:02:10 +0000] "GET /upstreams/f900e11f-c76d-4a5c-9f8b-49842b5f6045/targets/active/ HTTP/1.1" 200 186 "-" "node-fetch/1.0 (+https://github.com/bitinn/node-fetch)"
127.0.0.1 - - [30/Jan/2018:16:02:10 +0000] "GET /upstreams/61595731-6ea4-4625-98b8-3aa74e8c2481/targets/active/ HTTP/1.1" 200 53 "-" "node-fetch/1.0 (+https://github.com/bitinn/node-fetch)"
127.0.0.1 - - [30/Jan/2018:16:02:10 +0000] "GET /upstreams/8482419d-792f-4c04-8dc0-39ae1606e020/targets/active/ HTTP/1.1" 200 185 "-" "node-fetch/1.0 (+https://github.com/bitinn/node-fetch)"
127.0.0.1 - - [30/Jan/2018:16:02:10 +0000] "GET /upstreams/3d7d0843-4f71-4e8e-8a0d-ba20df44ab09/targets/active/ HTTP/1.1" 200 184 "-" "node-fetch/1.0 (+https://github.com/bitinn/node-fetch)"
127.0.0.1 - - [30/Jan/2018:16:02:10 +0000] "GET /upstreams/4ad05110-7f28-4d1b-92a1-0c45496a0a3f/targets/active/ HTTP/1.1" 200 184 "-" "node-fetch/1.0 (+https://github.com/bitinn/node-fetch)"

The config file (json) is large, about 2k lines, with around 150 upstreams.

Issue Analytics

  • State:closed
  • Created 6 years ago
  • Comments:6 (2 by maintainers)

github_iconTop GitHub Comments

1reaction
gavinlovecommented, Jan 30, 2018

@cliftonc This is a problem with your setup, not Kongfig.

You should change your Postgres max_connections config this needs to be greater than the number of nginx worker_processes * worker_connections otherwise you might get this problem if you have lots of concurrent connections going through kong from actual users. Alternatively, reduce those numbers in your kong config to be below your Postgres max connections

0reactions
cliftonccommented, Jan 31, 2018

I’ll try that, thanks for your help and input, and for open sourcing this, it’s a huge help and fits our config as code model perfectly. We wouldn’t be experimenting with Kong if this didn’t exist.

Read more comments on GitHub >

github_iconTop Results From Across the Web

"Too many concurrent requests for user" error while calling ...
The 429 error "Too many concurrent requests for user" means that you are making to many parallel requests - either by performing simulatenous ......
Read more >
The API client is making too many concurrent requests, and ...
1 Answer ... "Error 429 Too Many Requests" is sent as a response when the user has exhausted the maximum number of API...
Read more >
There are too many concurrent requests to this API being ...
There are too many concurrent requests to this API being processed. Hello, I'm currently using the free tier of playfab. In my game,...
Read more >
Too many simultaneous requests to the protected endpoint
This means that you will need to wait for one request to finish completely before you can update another record.
Read more >
429 Error – Too Many Requests HTTP Code Explained
The most common cause of the 429 error is not having enough resources to handle so many concurrent requests. For example, if this...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found
Previous page

[Android] - Crashes when any accessiblity is used

Next page

kongfig - diff against API and automatically delete what's not on config