Dev mode on Next.js v12 breaks with combination of features

What version of Next.js are you using?

12.0.0 and 12.0.1

What version of Node.js are you using?

v16.13.0 and v14.18.1 and v12.22.7

What browser are you using?

Doesn’t apply, breaks on terminal

What operating system are you using?

Arch Linux and macOS Big Sur 11.6

How are you deploying your application?

next dev

Describe the Bug

next dev breaks on v12, but I’m not exactly sure why, I was able to create a minimal reproduction repo with a combination of features, but I tried reproducing the issue with almost every feature (externalDir, chakra-ui, monorepo, typescript of course) of the repository by itself and worked fine, so I’m not completely sure what breaks it

Expected Behavior

dev mode should work??, in v11 everything works fine

To Reproduce

https://github.com/PabloSzx/next-v12-bug

> next

ready - started server on 0.0.0.0:3000, url: http://localhost:3000
warn  - You have enabled experimental feature(s).
warn  - Experimental features are not covered by semver, and may cause unexpected or broken application behavior. Use them at your own risk.

event - compiled successfully in 3.2s (1515 modules)
wait  - compiling /_error...
event - compiled successfully in 322 ms (1516 modules)

<--- Last few GCs --->

[60697:0x4967150]    24587 ms: Scavenge 4046.5 (4123.3) -> 4046.0 (4131.3) MB, 5.2 / 0.0 ms  (average mu = 0.387, current mu = 0.119) allocation failure 
[60697:0x4967150]    24599 ms: Scavenge 4052.6 (4131.3) -> 4051.3 (4131.6) MB, 10.1 / 0.0 ms  (average mu = 0.387, current mu = 0.119) allocation failure 
[60697:0x4967150]    25112 ms: Scavenge 4053.6 (4131.6) -> 4052.5 (4152.6) MB, 511.8 / 0.0 ms  (average mu = 0.387, current mu = 0.119) allocation failure 


<--- JS stacktrace --->

FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
 1: 0xb02eb0 node::Abort() [node]
 2: 0xa181fb node::FatalError(char const*, char const*) [node]
 3: 0xced75e v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [node]
 4: 0xcedad7 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [node]
 5: 0xea5d75  [node]
 6: 0xeb544d v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [node]
 7: 0xeb814e v8::internal::Heap::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [node]
 8: 0xe7957a v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationType, v8::internal::AllocationOrigin) [node]
 9: 0x11f2d56 v8::internal::Runtime_AllocateInYoungGeneration(int, unsigned long*, v8::internal::Isolate*) [node]
10: 0x15e7759  [node]

[timneutkens]

This seems to be a particular case with Chakra UI and some other deps. As a temporary workaround you can disable esmExternals:

module.exports = {
  experimental: {
    esmExternals: false
  }
}

[sokra]

fix in on the way.

Here a little explanation what happend here:

Usually Next.js will not bundle node_modules in the server compiler. It will make them “external” and either require() or import() them. But it has some checks in place to check if a package can be made external. One of these check is to compare if require() resp. import() leads to the same result in Node.js (otherwise this would break in Node.js). For import "@chakra-ui/react" it first resolves it to node_modules/@chakra-ui/react/dist/esm/index.js. It checks if this is an ESM or CJS module by extension (.mjs) or "type": "module". That results in “This is not an ESM”. That seems incorrect, but note that we are performing that check from Node.js viewpoint and it only check these criteria and @chakra-ui/react is actually missing the "type": "module" here. So Next.js checks if require("@chakra-ui/react") would result in the same module, but it actually don’t do that. It would resolve to node_modules/@chakra-ui/react/dist/cjs/index.js. So Next.js concludes that this module can’t be made external. Mainly because the Node.js ESM flags are missing in this package. As fallback it tries to bundle the package. That’s not something we want, mainly for performance reasons.

But that sadly bundling the package also doesn’t work correctly in combination with other nested package being made externals. I think it has something to do with async modules and circular dependencies, but I’m still investigating. Actually it shouldn’t run into that case anyway.

---

The fix is that Next.js now detects that case and tries to fallback to the CJS version of the package first, before falling back to bundling the package.

---

Chakra UI Contributor here. Anything we can do on our side?

Yes. Please add a package.json with { "type": "module" } to dist/esm/package.json in your packages. That will flag these packages correctly for Node.js (and by using a nested package.json it will only flag that folder).

You can try to import your packages via Node.js to check if they are valid:

$ node -e "import('@chakra-ui/react')"
(node:19932) Warning: To load an ES module, set "type": "module" in the package.json or use the .mjs extension.
(Use `node --trace-warnings ...` to show where the warning was created)
.../node_modules/@chakra-ui/react/dist/esm/index.js:1
export * from "./chakra-provider";
^^^^^^

SyntaxError: Unexpected token 'export'

In addition to that file extensions are mandatory in Node.js ESM: https://nodejs.org/api/esm.html#mandatory-file-extensions

-export * from "./chakra-provider";
+export * from "./chakra-provider.js";
-export * from "./extend-theme";
+export * from "./extend-theme.js";
-export * from "./theme-extensions";
+export * from "./theme-extensions.js";