Coverage reports are not restored when cache is used

Current Behavior

If the cache for tests is enabled and after first run with coverage on the subsequent runs test coverage will not be restored even when passing flag --codeCoverage.

I’m not sure if this affects all the test runners but I experience this with @nrwl/jest.

Expected Behavior

When a test was ran with coverage report then repeated tests with coverage must restore coverage reports.

Steps to Reproduce

• Create or use existing NX repo with test runner used with @nrwl/jest
• Make sure that cache for tests is enabled in nx.json
• Execute the tests with coverage report nx test --codeCoverage
• Remove generated report folder (usually in coverage/**)
• Execute the tests with coverage again nx test --codeCoverage
• Assert that cached output was used and no coverage report is on file system.

NOTE: You may use my public repo for testing as it’s public and that is where I hit this error: https://github.com/gund/time-slots

Please also note that I’ve disabled the cache for tests for time being as my CI was failing due to missing coverage reports (failed job as example where the cached test output was used)

Failure Logs

When the cache is used for tests with coverage:

> @gund/time-slots@0.0.0 test:ci /home/travis/build/gund/time-slots
> nx test --codeCoverage
> nx run time-slots:test --codeCoverage 
>  NX   NOTE  Cached Output:
PASS time-slots packages/time-slots/src/lib/time-slots-excluder.spec.ts
PASS time-slots packages/time-slots/src/lib/time-slots-generator.spec.ts
Test Suites: 2 passed, 2 total
Tests:       3 passed, 3 total
Snapshots:   0 total
Time:        2.613 s
Ran all test suites.
>  NX   CLOUD  Results for time-slots were retrieved from Nx Cloud. See run details at https://nx.app/runs/G0r5rKk5abD

The folder with coverage report is not recovered and codecov tool fails:

> codecov -p coverage/packages/time-slots --disable=gcov
  _____          _  
 / ____|        | |  
| |     ___   __| | ___  ___ _____   __  
| |    / _ \ / _` |/ _ \/ __/ _ \ \ / /  
| |___| (_) | (_| |  __/ (_| (_) \ V /  
 \_____\___/ \__,_|\___|\___\___/ \_/  
                                v3.8.0
==> Detecting CI Provider
    Travis CI Detected
==> Configuration: 
    Endpoint: https://codecov.io
{
  commit: '6b383b7c6b0eca2a690ca1676ed04b762d638454',
  branch: 'master',
  package: 'node-v3.8.0'
}
==> Building file structure
internal/fs/utils.js:269
    throw err;
    ^
Error: ENOENT: no such file or directory, scandir '/home/travis/build/gund/time-slots/coverage/packages/time-slots'

Environment

>  NX  Report complete - copy this into the issue template

  nx : Not Found
  @nrwl/angular : Not Found
  @nrwl/cli : 10.3.1
  @nrwl/cypress : Not Found
  @nrwl/eslint-plugin-nx : 10.3.1
  @nrwl/express : Not Found
  @nrwl/jest : 10.3.1
  @nrwl/linter : 10.3.1
  @nrwl/nest : Not Found
  @nrwl/next : Not Found
  @nrwl/node : 10.3.1
  @nrwl/react : Not Found
  @nrwl/schematics : Not Found
  @nrwl/tao : 10.3.1
  @nrwl/web : Not Found
  @nrwl/workspace : 10.3.1
  typescript : 4.0.3

[un33k]

For those who end up on this page looking for an answer - NX 11-beta-16 works.

The following migration must succeed.

{
      "version": "11.0.0-beta.16",
      "description": "Adds `outputs` based on builders",
      "factory": "./src/migrations/update-11-0-0/add-outputs-in-workspace",
      "package": "@nrwl/workspace",
      "name": "add-outputs-in-workspace"
}

Then all your projects need to setup an “outputs” list, something like "outputs": ["coverage/<app-or-lib-path>", "<app-or-lib-path/reporters"].

Pretty much anything you want to push up to nx cloud cache and retrieve on cache hit can be put in the above list. I am sure NX docs will follow soon.

[Esmilis]

Actually if you add outputPath to test options in workspace.json, it will get cached and restored on subsequent builds

{"example-app":` {
      "root": "apps/example-app",
      ...
      "architect": {
        "build": {
         ...
        },
        "serve": {
         ...
        },
        "lint": {
         ....
        },
        "test": {
          "builder": "@nrwl/jest:jest",
          "options": {
            "outputPath": "coverage/apps/example-app", <----add this
            "jestConfig": "apps/example-app/jest.config.js"
          }
        }
      }

Keep in mind that cache is restored by folder, so don’t mix output from different apps/libs within the same one.