Next >12.0.1 causes memory leak after upgrading from Next 11

Verify canary release

• I verified that the issue exists in Next.js canary release

Provide environment information

$ next info
(node:39290) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

What browser are you using? (if relevant)

Chrome 100

How are you deploying your application? (if relevant)

custom express server running on GCP distributed across multiple containers running 1GB memory

Describe the Bug

We’re in a bit of a catch22 because we tried to upgrade to 12.0.0, which works, but then our Lighthouse scores started failing due to a security vulnerability in 12.0.0. I don’t know if y’all ever fixed that but something was done in 12.0.5 it seems based on the change log, so we tried that, which created another issue: memory leaks.

After upgrading to 12.0.5 a new issue occurred where the prod optimized build app simply crashes due to OOM errors:

<--- Last few GCs --->
12:22:59.957 dev-dev-uc1-gke next-4072-webserver
12:22:59.958 dev-dev-uc1-gke next-4072-webserver [27:0x7f24d13882c0]   755827 ms: Mark-sweep (reduce) 770.6 (773.1) -> 770.3 (774.9) MB, 998.9 / 0.0 ms  (average mu = 0.100, current mu = 0.001) allocation failure scavenge might not succeed
12:22:59.958 dev-dev-uc1-gke next-4072-webserver [27:0x7f24d13882c0]   756837 ms: Mark-sweep (reduce) 771.3 (776.9) -> 771.1 (777.6) MB, 1009.1 / 0.0 ms  (average mu = 0.052, current mu = 0.001) allocation failure scavenge might not succeed
12:22:59.958 dev-dev-uc1-gke next-4072-webserver
12:22:59.958 dev-dev-uc1-gke next-4072-webserver
12:22:59.958 dev-dev-uc1-gke next-4072-webserver <--- JS stacktrace --->
12:22:59.958 dev-dev-uc1-gke next-4072-webserver
12:22:59.958 dev-dev-uc1-gke next-4072-webserver FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
12:23:00.016 dev-dev-uc1-gke next-4072-webserver error Command failed with signal "SIGABRT".

I tried 12.1 and even canary releases to see if anything worked itself out. Same problem. I tracked down a memory leak with growing Promises and Arrays within webpack runtime and cjs-loader and tslib (whatever that means) when building prod build locally and running node --inspect:

In the end, I found 12.0.1 to be the only version after 12.0.0 with no leak; going to 12.0.2 starts the memory leak and crashes the app in the same way.

The prime suspects are the following deps we have installed based on their interaction with promises and arrays:

• next >=12.0.2
• node 14.17.6
• express 4.17.1
• babel?
• axios?
• styled-components 5.3.3
• react 17.0.2
• redux
• typescript 4.3.2

Already tried upgrading node to latest LTS (16), still doesn’t work. Local dev runs “fine” albeit slow, but local builds don’t crash. Only prod optimized builds

Expected Behavior

App is expected to NOT crash, so I upgraded my personal website to 12.1, which is hosted on Vercel and has less dependencies than the enterprise, e-commerce site I am working on. Well, it is fine and running great, so makes me think the issue is more complex possibly around the use of a custom server in the enterprise app and/or Next12 isn’t playing nice with some node deps.

When I ran this in 12.0.0 and 12.0.1 the memory leak is non-existent:

It is not until 12.0.2 that we experience the memory leak! and I’ve gone through your change logs and nothing looks suspicious.

To Reproduce

Unable to reproduce with custom server next example, so it’s likely because we’re using an enterprise app and something (ie other dependencies) isn’t playing nice with next12 and would like to understand what could be causing that.

I can only assume something around SWC would have changed the build process so when it runs in node it gets wonky? Just speculating at this point because we can’t upgrade to next12 at all.

But in our enterprise app, I simply do the following to reproduce consistently every time:

• Next >12.0.1
• yarn build (uses babel, not SWC)
• NODE_ENV=production node --inspect build/server/index.js

[moshie]

Confirmed here early reports are showing that 12.3.0 has fixed the leak our end 🎉

[timneutkens]

@timneutkens awesome thanks so much for that explanation! I trust that y’all have the right tests in place and it’s hard to know what enterprise, custom apps are doing with your framework. I get it.

Considering that the esmExternals: false change seems to help most people out of this OOM issue, should we add it to docs as a last resort “backwards compat” config change? For example, in Next12 upgrade guide:

“if you experience [this OOM error] first try setting esmExternals: false since it was turned on by default in Next12. Doing this will…”?

What do you think?

I’d prefer to fix the OOM even if it’s not specifically in Next.js through a combination of deps. That flag will definitely go away in the future. Main problem so far is that there is no reproduction provided that would allow us to spend time on it as I’d be happy to have someone on our team look at it if one is provided.

I’ve spent so much time already, so I may as well work on a solid reproduction outside a large enterprise app 😅 The problem is I don’t know what in our npm spaghetti is causing the conflict with Next12, so hard to reproduce with only a handful of deps in an isolated sandbox. Will try my best…

That’d be great! 🙏