Parallel builds w/ code coverage fail (random test each time)

Logs and screenshots Please provide debug logs by running Cypress from the terminal with DEBUG=code-coverage environment variable set, see the Debugging section of the README file.

See comment below

Versions

• What is this plugin’s version? If this is NOT the latest released version can you try the latest version, please?

"@cypress/code-coverage": "^3.8.1"

• If the plugin worked before in version X, but stopped after upgrading to version Y, please try the released versions between X and Y to see where the breaking change was.
• What is Cypress version?

"cypress": "4.10.0"

• What is your operating system?

Whatever linux version is in cypress/included Docker images.

• What is the shell?

sh

• What is the Node version?

12.14.1 (Using Docker image cypress-included:4.10.0)

• What is the NPM version?

🤷

• How do you instrument your application? Cypress does not instrument web application code, so you need to do it yourself.

Use craco to customize create-react-app’s webpack config to add the babel-plugin-instanbul plugin. Similar to what is done by instrument-cra.

I am now using the latest @cypress/instrument-cra (v1.3.1) and still seeing this error.

• When running tests, if you open the web application in regular browser, and open DevTools, do you see window.__coverage__ object? Can you paste a screenshot?
• Is there .nyc_output folder? Is there .nyc_output/out.json file. Is it empty? Can you paste at least part of it so we can see the keys and file paths?
• Do you have any custom NYC settings in package.json (nyc object) or in other NYC config files

  "nyc": {
    "extends": "@istanbuljs/nyc-config-typescript",
    "all": true,
    "exclude": [
      "cypress/**",
      "**/__fixtures__/**",
      "**/__mocks__/**",
      "**/__tests__/**"
    ],
    "report-dir": "./coverage-cypress",
    "temp-dir": "./coverage-cypress/.nyc_output",
    "reporter": [
      "text-summary",
      "json",
      "html"
    ]
  }

• Do you run Cypress tests in a Docker container?

Yes, custom image based on cypress-included:4.10.0.

Describe the bug

I have a total of 113 Cypress tests. One random test (always just one, always different than before) will fail with an error like this:

      CypressError: `cy.task('coverageReport')` failed with the following error:
 
 > Unexpected end of JSON input
 
 Because this error occurred during a `after all` hook we are skipping all of the remaining tests.

or

      CypressError: `cy.task('coverageReport')` failed with the following error:
 
 > Unexpected string in JSON at position 1904650
 
 Because this error occurred during a `after all` hook we are skipping all of the remaining tests.

Update: Seems to be related to parallel Cypress runs. Fairly consistently fails for me when running with parallelization, but succeeds when using a single (non-parallel) job.

[jhpedemonte]

We figured it out: the issue was that we had each parallel instance of Cypress tests writing to the same code coverage directory. This would cause conflicts and errors as they each tried to read/write to the same files.

The correct way to handle this for parallel Cypress runs is to have each individual Cypress task write to a unique directory, then merge the reports into one final report using nyc merge or nyc report.

[panduka777]

@jhpedemonte How did you make individual cypress tasks write to unique directories?