Node-API/`LibraryEngine`: `$disconnect` does not free up memory / kill engine

When Node-API/engineType=library is active, calling $disconnect() does not free up the memory the instance of the Engine is using. This is because the engine is not killed as it is/was with the BinaryEngine (child process was completely killed on stop()), but only the disconnect is sent to the Engine. Then when another $connect or Query (which implicitly connects) comes in, the same instance of the QueryEngine is used again (vs. a newly started one, as with the binary).

If you (unrealistically) create many PrismaClient instances and just $disconnect them after use, this leads to growing memory usage until the script ends.

Context:

• Observation made during investigation of https://github.com/prisma/prisma/issues/8989, https://github.com/prisma/prisma/issues/8989#issuecomment-909494922
• Reproduction: https://github.com/janpio/prisma-node-api-memory-investigation + https://github.com/janpio/prisma-node-api-memory-investigation/actions/runs/1187427937
• Hacky LibraryEngine that changes this: https://github.com/prisma/prisma/pull/9041
• Modified reproduction with this engine showing different properties: https://github.com/janpio/prisma-node-api-memory-investigation/actions/runs/1187430271

Reproduction above, running for 60 minutes:

• Currently: 25467 rss 1605.77 MB
• With library engine deleted on $disconnect: 43100 rss 288.34 MB

We think this is a problem, but not of highest priority and can be solved down the line.

[janpio]

Yes, that would be awesome. It might be related to the original problem I describe above, but is fundamentally a different area. If your reproduction helps understanding that problem in isolation, that would be amazing.

That cleanup code is pretty neat by the way - although a tiny bit scary. Good that you scoped it to the current user etc.

[haleksandre]

I found out that the connections goes into a “idle” state. Either $disconnect isn’t working properly or it is putting the connection in “idle” state.

I found a workaround to keep the connections down with a SQL script, which you either call within the middleware or teardown with the help of the $executeRawUnsafe function.

This is the SQL script:

WITH inactive_connections AS (
  SELECT
    pid,
    rank() over (partition by client_addr order by backend_start ASC) as rank
  FROM
    pg_stat_activity
  WHERE
    -- Exclude the thread owned connection (ie no auto-kill)
    pid <> pg_backend_pid( )
  AND
    -- Exclude known applications connections
    application_name !~ '(?:psql)|(?:pgAdmin.+)'
  AND
    -- Include connections to the same database the thread is connected to
    datname = current_database()
  AND
    -- Include connections using the same thread username connection
    usename = current_user
  AND
    -- Include inactive connections only
    state in ('idle', 'idle in transaction', 'idle in transaction (aborted)', 'disabled')
  AND
    -- Include old connections (found with the state_change field)
    current_timestamp - state_change > interval '10 seconds'
)

SELECT
  pg_terminate_backend(pid)
FROM
  inactive_connections
WHERE
  rank > 1 -- Leave one connection for each application connected to the database;

// Teardown
await ctx.prisma.$executeRawUnsafe(`
  WITH inactive_connections AS (
    SELECT
      pid,
      rank() over (partition by client_addr order by backend_start ASC) as rank
    FROM
      pg_stat_activity
    WHERE
      -- Exclude the thread owned connection (ie no auto-kill)
      pid <> pg_backend_pid( )
    AND
      -- Exclude known applications connections
      application_name !~ '(?:psql)|(?:pgAdmin.+)'
    AND
      -- Include connections to the same database the thread is connected to
      datname = current_database()
    AND
      -- Include connections using the same thread username connection
      usename = current_user
    AND
      -- Include inactive connections only
      state in ('idle', 'idle in transaction', 'idle in transaction (aborted)', 'disabled')
    AND
      -- Include old connections (found with the state_change field)
      current_timestamp - state_change > interval '10 seconds'
  )

  SELECT
    pg_terminate_backend(pid)
  FROM
    inactive_connections
  WHERE
    rank > 1 -- Leave one connection for each application connected to the database;
`);

This cleans up all the “idle” connections that has been idling for 10 seconds or more.

Should I open another issue so someone could try to replicate & investigate to see if it is indeed a bug with $disconnect?