Improve test performance

Hey there,

We’ve been using this package for quite some time now and it has improved the development experience at our company a lot! However, as our project gets larger and larger, one thing that bothers us is the performance of unit tests. Our project is medium-sized with about 75 libraries and 3 apps (Angular and NestJS) and about 350 test suites.

It takes about 12 minutes on my machine (windows, very decent specs) to run all tests using nx affected:test --all --parallel. In our CI environment (Linux) and on laptops this can take quite a bit longer, around 20 to 25 minutes. By using nx affected, we can run the unit tests for PRs in a few minutes, but for bigger PRs with more files, the time it takes to run the tests increases rapidly.

NOTE: If this is something nobody else is experiencing, feel free to close this issue right away (please comment on how you achieved good test performance).

After looking into a lot of issues, we’re pretty confident that ts-jest and typescript compilation time are the main contributing factors. It takes between 10-20 seconds for each library just to get jest to start running the tests. Once jest is running, the tests are fast. Using the fix from #1299 improved performance quite a bit (around 20-30%), but this still is slow for this amount of tests. kulshekhar/ts-jest#1115 is probably part of the reason.

We are still very confident that re-compiling all necessary files for each library individually is not a good approach for unit testing. That’s why we decided to try to run all our unit tests together. We altered the global jest.config.js and are now able to run all unit tests with the Jest CLI in a single test run. The time it takes to run all those tests is about 60 seconds on my pc and 90-100 seconds on a laptop. I did not yet test the speed in our CI environment. That’s a massive speedup (over 10x faster), but could still be improved upon, and that’s what this issue is actually about.

My suggestion is to alter nx affected:test to collect all the projects that have changed, then divide them by their needed jest setup (jest-preset-angular with JSDOM, just JSDOM or Node) and run those tests at once. This could be done by adding an entry to the testMatch array in the jest config dynamically for every library and would result in a maximum of three test runs.

EDIT: After investigating a bit further, Jest supports a config entry called projects since Jest v20. This is a perfect fit for this problem. If the global jest config was altered for every new project generated (just like tsconfig.json and nx.json), it would be possible to run jest from the root directly through the CLI. This would also allow configuring each project individually as needed through its own jest.config.js and to run ng test <library> just like now. In addition, I would remove the hack to alter the jest config in a script and instead change the schematic that is used to generate a new angular library to add the config needed for ‘jest-preset-angular’ to the libraries’ jest.config.js.

nx affected:test would then be even easier to implement then in my previous suggestion. Just replace the projects entry in the global jest config by an array of changed project paths.

Pros:

• Unit tests run a lot faster
• Jest config is cleaner and more exposed
• Jest CLI can be used to run unit tests
• VSCode-Jest can be used to run unit tests
• The coverage report shows the total coverage

Cons:

• Parsing failed projects is probably a bit more difficult to implement

Expected Behavior

Be able to run unit tests in 1-2 minutes for small and medium-sized projects and even less for PRs.

Current Behavior

It takes about 12 minutes on my machine (windows, very decent specs) to run all tests. In our CI environment (Linux) and on laptops this can take quite a bit longer, around 20 to 25 minutes.

Context

Please provide any relevant information about your setup:

@nrwl/angular : 8.11.0 @nrwl/cli : 8.11.0 @nrwl/cypress : 8.11.0 @nrwl/jest : 8.11.0 @nrwl/linter : 8.11.0 @nrwl/nest : 8.11.0 @nrwl/node : 8.11.0 @nrwl/tao : 8.11.0 @nrwl/workspace : 8.11.0 typescript : 3.4.5

jest-preset-angular: 7.1.1 jest: 24.8.0 ts-jest: 24.3.0

[luchsamapparat]

I started running the tests with the --run-in-band flag a while ago. At least under Windows that was way faster. However, this was already the case before caching was introduced.

[ahnpnl]

FYI: https://github.com/kulshekhar/ts-jest/pull/1549 will be in alpha version of ts-jest (possibly today). Anyone who is using ts-jest please help to test the alpha version and give us some feedbacks for https://github.com/kulshekhar/ts-jest/issues/1115