Initial build time increases 2-3x from webpack 4

My team is in the process of upgrading from Webpack 4 to Webpack 5, and we’re encountering very significant performance problems. For reference, our development mode bundle consists of

• 6788 .js, .ts, and .tsx files (babel-loader), including 214 dynamic import() operators.
• 1 web worker (the above + worker loader)
• 1 .html template (html-loader)
• 322 .slim templates (the above + slim-loader)
• 12 CSS files (style-loader + css-loader)
• 324 SASS files in .scss format (the above + resolve-url-loader, sass-loader, and @epegzz/sass-vars-loader)
• 12 SASS module files (the above + @teamsupercell/typings-for-css-modules-loader)
• 1 csv file (type: asset/source)
• 149 images (type: asset)

And numerous node_modules. Our build time on webpack 4 with EventHooksPlugin and devtool: eval is about 60 seconds. Rebuilds with cached babel-loader artifacts are about 46 seconds. Incremental rebuilds in watch mode are about 3.4 seconds.

On our Webpack 5 branch, we have upgraded our configuration to be compliant with new Webpack 5 idioms in almost every regard, including asset modules, replacing various plugins with built-in settings e.g. (DefinePlugin -> optimization: nodeEnv, CleanWebpackPlugin -> output: clean), disabling the babel-loader cache in favor of webpack filesystem caching, and adopting compiler.close() for usage in the node API. The single exception is worker-loader, which we have had to retain in lieu of new Worker() due to https://github.com/webpack/webpack/issues/13801.

With default optimization settings and filesystem caching enabled, our build time is now about 131 seconds. Cached rebuilds are about 11.5 seconds. Incremental rebuilds in watch mode are about 4.3 seconds.

With the following optimizations disabled

• providedExports
• usedExports
• sideEffects
• innerGraph
• mergeDuplicateChunks
• removeEmptyChunks

we were able to reduce cached rebuild times to 10.5 seconds and watch mode rebuild times to about 3.7 seconds. The initial build time was unaffected.

Unfortunately, when we attempted to profile webpack with both ProfilingPlugin and --inspect-brk, the resulting profiles were too large (~800 mb) to be parsed by the Chrome developer tools, so we don’t yet have a clear signal on where the degraded performance is coming from.

In order to rule out any of our loaders and plugins, we disabled everything but babel-loader by reassigning all non-js types to type: asset/source. This non-operable bundle, effectively a skeleton of our JS module’s dependency graph, built in 95 seconds on Webpack 5, indicating that the source of the problem lies in Webpack’s module resolution or chunking behavior.

Gaining Webpack 5’s advanced tree-shaking features is very important to us, as is remaining on the leading edge of Webpack development and in conversation with the Webpack community. A certain amount of build-time performance cost in production mode is an acceptable trade-off. However, the prospect of a 2.15x increase in development mode cold start build times is a tough pill to swallow.

Although we are somewhat mollified by the very good performance with file system caching, we are unable to take advantage of this across the board due to our use of hermetic, containerized environments in our CI system, and would appreciate help investigating, insight into, and fixes for the severe regression in cold start build times.

[AprilArcus]

I’m an idiot; I removed thread loader during the upgrade.

[AprilArcus]

While I work on redacting these logs, I noticed that when I set parallelism: 1 for profiling, the difference in build times collapsed and webpack 4 entered the ~130s ballpark as well.