Build performance decreased from v12.2.x to v13.1.1 (process assets / seal compilation) if Bootstrap is imported in styles.scss

Which @angular/* package(s) are the source of the bug?

angular_devkit/build_angular

Is this a regression?

Yes

Description

I just upgraded a pretty big private project from v12 to v13 and the build time increased x10, can’t really find why. Just to be sure that it was Angular upgrade I tried to downgrade again and the performance restored as expected. I post here a part of the verbose log of the build, in particular note the extreme difference between v12 and v13 in assets time / seal compilation:

v12 (edit)

...
<t> record hash: 0.12885 ms
<t> module assets: 0.478536 ms
<t> create chunk assets: 6.774782 ms
<t> process assets: 30310.76689 ms <--------

LOG from webpack.Compiler
<t> make hook: 18985.143228 ms
<t> finish make hook: 0.206746 ms
<t> finish compilation: 72.86418 ms
<t> seal compilation: 32318.405325 ms <----------
<t> afterCompile hook: 0.280398 ms
...

v13

...
<t> record hash: 0.130436 ms
<t> module assets: 0.797611 ms
<t> create chunk assets: 6.450294 ms
<t> process assets: 302297.57658 ms <-----------

LOG from webpack.Compiler
<t> make hook: 21553.025473 ms
<t> finish make hook: 0.179688 ms
<t> finish compilation: 86.421 ms
<t> seal compilation: 304301.925952 ms <-----------
<t> afterCompile hook: 0.207707 ms
...

tsconfig.json:

/* To learn more about this file see: https://angular.io/config/tsconfig. */
{
  "compileOnSave": false,
  "compilerOptions": {
    "baseUrl": "./",
    "outDir": "./dist/out-tsc",
    "forceConsistentCasingInFileNames": true,
    //"strict": true,
    "noImplicitOverride": true,
    //"noPropertyAccessFromIndexSignature": true,
    "noImplicitReturns": true,
    "noFallthroughCasesInSwitch": true,
    "sourceMap": true,
    "declaration": false,
    "downlevelIteration": true,
    "experimentalDecorators": true,
    "moduleResolution": "node",
    "importHelpers": true,
    "target": "es2017",
    "module": "es2020",
    "lib": [
      "es2020",
      "dom"
    ]
  },
  "angularCompilerOptions": {
    "enableI18nLegacyMessageIdFormat": false,
    "strictInjectionParameters": true,
    "strictInputAccessModifiers": true,
    "strictTemplates": true
  }
}

Any help to debug this issue deeply? What other tests could I do to find out more?

Thanks

Please provide a link to a minimal reproduction of the bug

https://github.com/angular/angular/issues/44555#issuecomment-1005045938

Please provide the exception or error you saw

Performance issue

Please provide the environment you discovered this bug in (run ng version)

Angular CLI: 13.1.2
Node: 16.13.1
Package Manager: npm 8.3.0
OS: linux x64

Angular: 13.1.1
... animations, common, compiler, compiler-cli, core, forms
... language-service, platform-browser, platform-browser-dynamic
... router

Package                         Version
---------------------------------------------------------
@angular-devkit/architect       0.1301.2
@angular-devkit/build-angular   13.1.2
@angular-devkit/core            13.1.2
@angular-devkit/schematics      13.1.2
@angular/cli                    13.1.2
@schematics/angular             13.1.2
rxjs                            7.4.0
typescript                      4.5.4

Anything else?

The issue is not present on development build (-c development) The issue is present in v13.0.0 too

[DavideViolante]

So, after several tests, I found out the root cause.

How I found it

To find it out, as I said, I’ve 2 projects, one of which is a more complex version of the other. The simple project didn’t have any issue with v13, while the complex one does. So I started from the complex project and I went back hundreds of commits (but keeping v13 in package.json), to discover which change caused the build time to increase a lot.

Root cause

I found out that it’s related to SCSS and Bootstrap, in particular if you import Bootsrap in styles.scss as they say here, the production build times increases by several factors.

How to reproduce

1. Clone this repo
2. Run ng config cli.cache.enabled false to disable the caching on build (important to compare different v13 builds)
3. (optional) Run npm i then npm run build to annotate the normal build timings and be able to compare later
4. Edit styles.scss and add this line @import '../node_modules/bootstrap/scss/bootstrap'; on top
5. Run npm i then npm run build and get a coffee meanwhile
6. Annotate the new build timings (below are mine)

Compare with v12

1. After the steps above, on my repo run git reset --hard HEAD~16 to revert to v12
2. Repeat step 4 above
3. Run npm i then npm run build and annotate the timings and compare them with v13 (below are mine)

Differences

v13 no Bootstrap import on styles.scss cache disabled
Build at: 2022-01-04T17:23:04.303Z - Hash: abc86b58f4200b3d - Time: 88389ms

v13 Bootstrap import on styles.scss cache disabled
Build at: 2022-01-04T17:34:43.699Z - Hash: 474fdca2989cd28d - Time: 627843ms <-----

v12 no Bootstrap import on styles.scss cache disabled
Build at: 2022-01-04T17:51:44.189Z - Hash: 59b34432059ba04e80f6 - Time: 88004ms

v12 Bootstrap import on styles.scss cache disabled
Build at: 2022-01-04T17:49:26.608Z - Hash: 0768bac24ea295cba933 - Time: 153772ms <----- increased but not as much as v13

[DavideViolante]

I’m sorry I picked the wrong part of the logs from v12, I edited first post, the times are around 30s, compared to 300s in v13.

Here the full ng build verbose logs: v12: https://pastebin.com/XAxkHw5A v13: https://pastebin.com/6mme47Lr

I also edited the previous post to provide the full cpu-prof files.

Just a side note: while building v13 the system is not usable due to 100% CPU, not happening “so strong” with v12.