Incremental build time increased drastically since v6.3

Describe the bug Incremental build time was increased after updating to v6.3. It was about 4s and becomes about 15s.

To Reproduce TBD

System

Environment Info:

  System:
    OS: macOS 11.4
    CPU: (16) x64 Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz
  Binaries:
    Node: 14.17.0 - /usr/local/bin/node
    Yarn: 2.4.2 - /usr/local/bin/yarn
    npm: 6.14.13 - /usr/local/bin/npm
  Browsers:
    Chrome: 93.0.4549.4
    Edge: 91.0.864.59
    Safari: 14.1.1

Additional context I can’t create a reproduction for now, because it’s quite tricky. But I’ll try to describe the investigation that was done.

First of all, using SpeedMeasureWebpackPlugin was determined, that the culprit is DocgenPlugin:

 SMP  ⏱  Plugins
DocgenPlugin took 10.71 secs

Then I excluded it from the preview build, and incremental build time returned to normal ~4s. Great.

react-docgen-typescript-plugin v0.6.2 was used in Stoybook v6.2.9. Storybook v6.3 depends on v1.0.2-canary.XXX.

Running Storybook with DEBUG=docgen:* variable, I saw that it tries to match all 3000+ modules on each rebuild:

webpack building...
  docgen:exclude Module not matched in "include": /XXX/.yarn/$$virtual/XXX-0.0.0.zip/node_modules/main.js +0ms
  docgen:exclude Module not matched in "include": /XXX/.yarn/$$virtual/XXX-0.0.0.zip/node_modules/main.js +0ms
  docgen:exclude Module not matched in "include": /XXX/.yarn/cache/XXX-0.0.0.zip/node_modules/main.js +0ms
  ...
  docgen:exclude Module not matched in "include": /XXX/.yarn/$$virtual/XXX-0.0.0.zip/node_modules/main.js +0ms
  docgen:exclude Module not matched in "include": /XXX/.yarn/cache/XXX-0.0.0.zip/node_modules/main.js +0ms
  docgen:exclude Module not matched in "include": /XXX/.yarn/$$virtual/XXX-0.0.0.zip/node_modules/main.js +0ms
  docgen:exclude Module not matched in "include": /XXX/.yarn/unplugged/XXX-0.0.0.zip/node_modules/main.js +2ms
  ...
  docgen:exclude Module not matched in "include": /XXX/some/local/module/XXX +0ms
  docgen:exclude Module not matched in "include": /XXX/some/local/module/XXX +1ms
  docgen:exclude Module not matched in "include": /XXX/some/local/module/XXX +0ms
  ...

Downgraded to react-docgen-typescript-plugin and ran again with DEBUG option. It logs the same modules, but with other “reasons”. And it goes fast (incremental build time is normal).

  docgen:exclude Ignoring un-built module: /XXX/.yarn/$$virtual/XXX-0.0.0.zip/node_modules/main.js +0ms
  ...
  docgen:exclude Ignoring un-built module: /XXX/.yarn/unplugged/XXX-0.0.0.zip/node_modules/main.js +2ms
  docgen:exclude Ignoring un-built module: /XXX/some/local/module/XXX +0ms
  ...

Seems like the reason is in this PR https://github.com/hipstersmoothie/react-docgen-typescript-plugin/pull/33 Prior to that, the iteration over modules was with a simple check for !module.built, and goes very fast. For now, it always matches with micromatch. I couldn’t understand why those checks were removed, but maybe @bebraw help with it.

Hope this info will help. I can create a separate issue in the react-docgen-typescript-plugin repo if it needs to.

[alexander-akait]

https://github.com/webpack/webpack/pull/7933

module.built has be removed in favor of Compilation.builtModules

⭐

[bebraw]

Prior to that, the iteration over modules was with a simple check for !module.built, and goes very fast. For now, it always matches with micromatch. I couldn’t understand why those checks were removed, but maybe @bebraw help with it.

Ah, I see now. I missed that check for the new webpack 5 portion. Thanks for bringing this up, I’ll add the check there.