API calls failing with Error querying the database: db error: FATAL: remaining connection slots are reserved for non-replication superuser connections

Bug description

I have been running my app on GCP that connects to a postgres instance also running in GCP, for the past few months. However, since yesterday, my REST API calls are only working intermittently.

I have been getting this error about 50% of the time. Other times, it seems to work fine: Error querying the database: db error: FATAL: remaining connection slots are reserved for non-replication superuser connections

I have been initializing one prisma instance per endpoint (total 8 endpoints) and till date have had zero issues using connection_limit=80&pool_timeout=50 on the DB string.

I am not sure if it is from have idle open connections that aren’t closing or if it is from my prisma queries or some other issue. Can you kindly advise me on how to fix this issue? Any help would be much appreciated. I am unable to reproduce this issue as it is quite sporadic. Thank you

Default
2021-05-17 17:10:00.614 GMTInvalid `prisma.instance.findMany()` invocation:
Default
2021-05-17 17:10:00.614 GMT
Default
2021-05-17 17:10:00.614 GMT
Default
2021-05-17 17:10:00.614 GMT Error querying the database: db error: FATAL: remaining connection slots are reserved for non-replication superuser connections
Default
2021-05-17 17:10:00.614 GMT at cb (/server/models/prisma-client/runtime/index.js:79166:17)
Default
2021-05-17 17:10:00.614 GMT at processTicksAndRejections (internal/process/task_queues.js:97:5)
Default
2021-05-17 17:10:00.614 GMT(node:14) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 4)
Default
2021-05-17 17:12:00.721 GMT(node:14) UnhandledPromiseRejectionWarning: Error:
Default
2021-05-17 17:12:00.721 GMTInvalid `prisma.events.findMany()` invocation:
Default
2021-05-17 17:12:00.721 GMT

Here is an example of the an endpoint where prisma instance gets initiated:

require('../../config/config');
const express = require('express');
const router = express.Router();

const prisma = require('../models/prismaClient.js');
const sendEmailSummary = require('../functions/sendEmailSummary');

router.post('/', async (req, res) => {
    await sendEmailSummary(prisma);
    res.sendStatus(200);
});
  

module.exports = router;

And here is the prisma client:

require('../../config/config');
const { PrismaClient } = require('./prisma-client');

const url = process.env.DATABASE_URL;
let prisma;

if (process.env.NODE_ENV === "production") {
  prisma = new PrismaClient({
    datasources: { db: { url } },
  })
} else {
  if (!global.prisma) {
    global.prisma = new PrismaClient({
      datasources: { db: { url } },
    })
  }
  prisma = global.prisma
}

module.exports = prisma;

How to reproduce

Expected behavior

Should be returning values from database

Prisma information

Environment & setup

• OS: MacOS
• Database: PSQL
• Node.js version: 14
• Prisma version: 2.18.0

Prisma Version

2.18.0

[janpio]

Ok then this is the reason for your problem.

Maybe observe the process list to see how it goes from 1 to 2 (or whatever starts that increase) and see if you can identify the action that does that. Something you are doing - or your code is - has that effect and should be fixed.

[shwethashyam1]

I usually check the processes after encountering the error and it is around 50+. Then I terminate all of them locally and wait for the error to happen again - by then the # processes would have increased to the same.