NextJs compiling extremely slow

What version of Next.js are you using?

11.1.2

What version of Node.js are you using?

14.18.0

What browser are you using?

Chrome

What operating system are you using?

Windows 10

How are you deploying your application?

AWS ECS

Describe the Bug

I’ve been using NextJs for years and recently it has been very hard to work with because very slow in development. After npm run dev, I go to localhost:3000. From there the page can take up to 60 seconds to display. Then when the first page finally displays, each code change fast refresh or page transition SSR (compilation) takes between 15-20 seconds, sometimes more than 30 seconds, and sometimes it even doesn’t work so I have to refresh the page.

• I read that on windows, windows defender could be blocking so I did this, which didn’t help: https://github.com/vercel/next.js/issues/12797#issuecomment-660225689
• I use webpack 5
• I read tailwind could be causing this kind of issue, but I don’t use it.

Expected Behavior

Page load + compilation should be faster.

To Reproduce

Unfortunately, I cannot send a reproducible UI because the project I work on is pretty big and the UI is under NDA.

Maybe without a reproducible UI, you guys have thoughts about how to improve/fix the compilation time on windows. Maybe some of you have faced the same problem and fixed it somehow. I’m all ears.

My package.json

{ “name”: “myname”, “version”: “0.1.0”, “private”: true, “scripts”: { “dev”: “next dev”, “build”: “next build”, “start”: “next start -p 3002” }, “dependencies”: { “@react-google-maps/api”: “^2.0.2”, “@sentry/browser”: “^6.7.2”, “@sentry/react”: “^6.7.2”, “@sentry/tracing”: “^6.7.2”, “@sentry/webpack-plugin”: “^1.15.1”, “@stripe/react-stripe-js”: “^1.4.0”, “@stripe/stripe-js”: “^1.13.1”, “accept-language-parser”: “^1.5.0”, “aws-sdk”: “^2.802.0”, “base-64”: “^1.0.0”, “cors”: “^2.8.5”, “dotenv”: “^8.2.0”, “hls.js”: “^0.14.16”, “iban”: “0.0.14”, “js-sha256”: “^0.9.0”, “libphonenumber-js”: “^1.9.11”, “localized-countries”: “^2.0.0”, “lodash”: “^4.17.21”, “moment”: “^2.29.1”, “next”: “^11.1.2”, “next-compose-plugins”: “^2.2.1”, “next-fonts”: “^1.5.1”, “next-images”: “^1.8.1”, “next-seo”: “^4.17.0”, “nookies”: “^2.5.0”, “platform”: “^1.3.6”, “qrcode”: “^1.4.4”, “randomstring”: “^1.1.5”, “react”: “^17.0.2”, “react-cropper”: “^2.1.4”, “react-datepicker”: “^3.3.0”, “react-device-detect”: “^1.15.0”, “react-dom”: “^17.0.2”, “react-dotdotdot”: “^1.3.1”, “react-ga”: “^3.3.0”, “react-geocode”: “^0.2.2”, “react-google-maps”: “^9.4.5”, “react-lines-ellipsis”: “^0.14.1”, “react-loading-skeleton”: “^2.2.0”, “react-query”: “^3.13.2”, “react-textarea-autosize”: “^8.3.2”, “sass”: “^1.29.0”, “stripe”: “^8.137.0”, “uuid”: “^8.3.1”, “video.js”: “^7.11.0”, “videojs-contrib-dash”: “^4.0.0” }, “devDependencies”: { “@svgr/webpack”: “^5.5.0” } }

[Carduelis]

I’ve experienced the issue with @mui/material and my own npm package @gorazdo/tomui The problem comes from a lot of modules to process (11k) like in this article Unfortunately, this solution doesn’t work. The problem is the regex (which is a completely mysterious thing for a non-rust developer. I’ve even tried Rust Regex online tool ), with no luck.

My Solution (3-5 times faster)

decreased 3-5 times dev build/rebuild/fast-refresh time decreased amount of modules by 10 times from 11k to 1k modules.

I’ve managed it using the following config:

    modularizeImports: {
      '@gorazdo/tomui': {
        transform: '@gorazdo/tomui/{{member}}',
      },
      '@mui/material': {
        transform: '@mui/material/{{member}}',
      },
      '@mui/icons-material/?(((\\w*)?/?)*)': {
        transform: '@mui/icons-material/{{ matches.[1] }}/{{member}}',
      },
    },

But I had to update imports manually for useTheme, ThemeProvider, styled, lighten and darken in my case

from:

import { Box, lighten, Typography, styled } from '@mui/material'
// for lighten and styled the regex didn't work neither the simple {{member}} way. so it is manually changed

to

import { Typography, Box } from '@mui/material'
import { styled, lighten } from '@mui/material/styles' 

To fix my library I had to use default export for each file which is going to be modularized.

Questions

1. How to deal with this regex to make it work with @mui/material ?
2. How to deal with modularization of files that have only named exports?
3. Can we automate modularization somehow?

[timneutkens]

Experiencing this issue?

To provide a trace which only includes metadata of the application:

• npm install next@canary (or yarn add next@canary)
• Run development like normal and reproduce the issue you’re experiencing
• Stop Next.js (generally ctrl + c)
• Send the file .next/trace file here or send it to https://twitter.com/timneutkens if you don’t want to share it publicly. You can use https://gist.github.com to create a private url for the file. - If you have a custom .babelrc please provide it - If you have a custom next.config.js please provide it - if you are using TailwindCSS please provide tailwind.config.js

⚠️ The metadata in .next/trace includes full file paths to all JS/CSS. It does not include the file contents. ⚠️ The main thing we need to help investigate your application is .next/trace, package.json and such are not helpful in investigating the issue.

Reply to earlier messages

but shouldn’t the potential solutions be posted here to help everyone that has this issue?

So far all I’ve done is help investigate the slowdowns for the people that reached out. There’s no specific solution as it highly depends on the app and I’m going to share the findings. . I’ve been working on a way that we can investigate these slowdowns without needing the application code itself but it requires a bunch of steps that up till a few days ago were quite involved.

Here’s a list of what I’ve found so far:

• @stevensturkop hasn’t provided a trace yet
• @PhillipLangMartinez
  • Fast Refresh is behaving as expected, no particular slowdown, will benefit from the work we’re doing for SWC and webpack optimizations though
  • The main problem they’re experiencing is about on-demand-entries. Next.js compiles the JS/CSS/etc for pages on-demand when you request them, so if you request http://localhost:3000/about Next.js will at that point start compiling the JS/CSS for pages/about.js, this is intentional to ensure that you can have thousands of pages in a Next.js app and it would not slow down your app, otherwise you’d pay this compilation penalty at bootup
    • This will also benefit from the work we’re doing integrating SWC with full backwards compat. I’ve had Phillip try the new flag and it’s resulted in at least 50% faster initial compilation and ~60% faster compiling of pages. Note that this highly depends on if you have a lot of node_modules or a lot of application code, the application code will be compiled faster but node_modules are not passed through babel currently so they are not affected by the work we’re doing to add SWC at this point in time.
  • In order to investigate this one further I’m adding timings for filesystem reads to find if that’s what is causing issues with a large amount of JS as they have a bunch of large node_modules like react-bootstrap included
• @kartikrao
  • 90% of the time spent compiling is related to TailwindCSS, it’s slowing down each CSS file by spending about 2 minutes and 30 seconds, it seems that the application is not using TailwindCSS JIT, so I’d recommend enabling that for all users that are using TailwindCSS: https://tailwindcss.com/docs/just-in-time-mode
• @f16z
  • Machine fs.readFile seems to be slow, it takes over 100ms per file, we expanded the tracing to include timing for individual files
  • Initial compile + compiling pages/index.js reads and compiles about 6867 files. What I’ve found is that 5557 of those are coming from @material-ui/icons/esm, assuming that the app is not using 5.5K icons I’d recommend changing how they are imported so that not all files have to be compiled. Based on debugging this trace we’re adding a separate chart that will allow us to visualize the dependency graph based on a trace file
• @billymcintosh can you provide a trace based on the above instructions