3.0 - custom reporters broken now that tests are run in isolation

Is this a Feature or Bug?

Bug

Current behavior:

Each test suite overrides the previous one’s results.

Desired behavior:

Before 3.0, results were aggregated and the final report contained all test suite results.

Steps to reproduce:

Run with several test suites:

cypress run --reporter junit --reporter-options "mochaFile=results.xml

After completion, results.xml only contains the last test suite results.

Versions

3.0

[eladmoshe]

If anyone tries running from the command line (or through package.json script), you cannot pass the [hash] param as --reporter-options argument. Instead, you need to add the following to your cypress.json:

 "reporter": "junit",
    "reporterOptions": {
        "mochaFile": "cypress/results/results-[hash].xml"
    }

[brian-mann]

@abataub how do you know when the run is finished? You use our module API, which will yield you all of the results or you simply chain something off of cypress run like cypress run && echo "finished"

@joshmccure nothing for this will be done with 3.0.2 because other than mochawesome we’re not seeing anything that is a fire that needs putting out. We have talked internally about exposing a service which will aggregate the results for you and spit out junit or mochawesome or whatever it is you want.

The reason this likely needs to be moved to the service level is because of parallelization releasing in 3.1.0. In that case the reports will not only be split out by spec file but they will be split out across machines and there won’t be an easy way to aggregate the results except if you have a service that knows everything about the run. That’s the logical place to put it. For now you can also see all of the results aggregated correctly in the Dashboard - although I do understand the desire to want to have junit or mochawesome reports.