Pooling + delete require.cache = connection leak

Environment

Knex version: 0.20.13 Database + version: PostgreSQL 12.2 OS: alpine

“Bug”

1. Explain what kind of behaviour you are getting and how you think it should do

Hi there, I am using knex with Next.js and noticed that my used connections from pg_stat_activity was always going up to a point where I would get an error message about them being exhausted.

It turns out this is caused by the way Next.js does server side hot module reloading: using delete require.cache means (cc @timneutkens do you confirm?). When the require cache is deleted and the module is re-required then the connection just leaks and will never be collected/terminated.

The interesting bit is that node-postgres pooling is not impacted by this issue and works fine.

1. Error message

error: sorry, too many clients already
    at Connection.parseE (/Users/vvo/Dev/knex-pool-require-cache-issue/node_modules/pg/lib/connection.js:600:48)
    at Connection.parseMessage (/Users/vvo/Dev/knex-pool-require-cache-issue/node_modules/pg/lib/connection.js:399:19)
    at Socket.<anonymous> (/Users/vvo/Dev/knex-pool-require-cache-issue/node_modules/pg/lib/connection.js:115:22)
    at Socket.emit (events.js:310:20)
    at addChunk (_stream_readable.js:286:12)
    at readableAddChunk (_stream_readable.js:268:9)
    at Socket.Readable.push (_stream_readable.js:209:10)
    at TCP.onStreamRead (internal/stream_base_commons.js:186:23) {
  name: 'error',
  length: 85,
  severity: 'FATAL',
  code: '53300',
  detail: undefined,
  hint: undefined,
  position: undefined,
  internalPosition: undefined,
  internalQuery: undefined,
  where: undefined,
  schema: undefined,
  table: undefined,
  column: undefined,
  dataType: undefined,
  constraint: undefined,
  file: 'proc.c',
  line: '361',
  routine: 'InitProcess'
}

1. Reduced test code

I have created an example here: https://github.com/vvo/knex-pool-require-cache-issue.

The interesting part being:

test-knex.js

const http = require("http");
const path = require("path");

const server = http.createServer(async (req, res) => {
  const db = require("./lib/db-knex");
  try {
    await db.query("SELECT 1;");
  } catch (e) {
    console.error(e);
    res.writeHead(500, { "Content-Type": "text/plain" });
    res.end("not ok, check the terminal of your Node.js server\n");
    return;
  }

  res.writeHead(200, { "Content-Type": "text/plain" });
  res.end("okay\n");
  delete require.cache[path.join(__dirname, "lib/db-knex.js")];
});

server.listen(3000);

lib/db-knex.js

const knex = require("knex");

const pg = knex({
  client: "pg",
  connection: process.env.DATABASE_URL,
});

module.exports = {
  query(sql) {
    return pg.raw(sql);
  },
};

Let me know what you think. It took me some time to pinpoint this but not sure we can do something about it. 🙏

[vvo]

For anyone reaching this issue and having connection leaks in dev mode of Next.js or any other situation where your db.js file is reloaded and connections are lost using Knex, you can solve it this way:

const knex = require("knex");
let pg;

if (process.env.NODE_ENV === "development") {
  global.pg =
    global.pg ||
    knex({
      client: "pg",
      connection: process.env.DATABASE_URL,
    });

  pg = global.pg;
} else {
  pg = knex({
    client: "pg",
    connection: process.env.DATABASE_URL,
  });
}

[icflorescu]

Had the same issue for the past 10 months while working on a large project with Next.js + Objection.js/Knex + PostgreSQL. Finally managed to solve it by implementing a similar hack:

/**
 * @returns {import('knex')}
 */
const getConnection = () => {
  if (PRODUCTION) return knex(DB);
  /**
   * This `global` hack solves a long-standing issue appearing in development more,
   * when successive hot-reloads lead to connections being leaked and eventually produce
   * an unrecoverable error
   */
  /* eslint-disable no-underscore-dangle */
  if (!global.__KNEX_DB_CONNECTION__) global.__KNEX_DB_CONNECTION__ = knex(DB);
  return global.__KNEX_DB_CONNECTION__;
  /* eslint-enable */
};

const db = getConnection();

export default db;