"Re-run failed jobs" will not work with a parallel workflow

After implementing a custom build ID to ensure I could re-run workflows which I’ve integrated with Cypress Dashboard and configured to run parallelly, I’ve run into an issue (oddly different than this one).

Here is my job FWIW:

- id: cypress-mocked-api-tests
  uses: cypress-io/github-action@v2
  with:
    wait-on: 'https://localhost:9001/index.js'
    start: npm run start:${{ env.NODE_ENV }}
    config-file: cypress/config/${{ env.NODE_ENV }}.json
    config: video=true,videoUploadOnPasses=false
    spec: '**/*.spec.ts'
    install: false
    record: true
    parallel: true
    group: 'Mocked-API'
    ci-build-id: ${{ needs.prepare.outputs.uuid }}

This job will load balance all my spec files across five containers under a “Mocked-API” group. This works great and I can re-run all jobs without issue.

On a recent run, one of the five containers failed because one test failed. I thought I’d test how “Re-run failed jobs” worked on just the failed container job. My hope/expectation was that it would be smart enough to know which spec files it ran when the entire workflow executed originally (which would have been six spec files which included 22 test) and run those. Instead it ran zero spec files and completed successfully. It seems like the matrix-level orchestration that is needed is not occurring when only a failed container job is re-run. It looks like someone else has run into this issue too and is trying to solve it by disabling the “Re-run failed jobs” option in Github (which doesn’t seem possible).

This is a fairly big problem because it resulted in the group (which I’ve configured as a status check in my trunk branch protection rule) passing and the PR being able to be merged when it had never successfully run all tests.

[ilovegithub2]

We are seeing exactly the same issue - rerunning failed jobs only will not run any tests but mark each as passed.

Here is our configuration

      - name: Run integration tests
        timeout-minutes: 20
        uses: cypress-io/github-action@v4
        env:
          CYPRESS_RECORD_KEY: ${{ secrets.CYPRESS_RECORD_KEY }}
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        with:
          ci-build-id: ${{ needs.prepare.outputs.uuid }}
          config: baseUrl=${{ format('https://pr{0}-www.build.{1}', github.event.number, env.CBR_PROJECT_DOMAIN) }}
          wait-on: ${{ format('https://pr{0}-www.build.{1}', github.event.number, env.CBR_PROJECT_DOMAIN) }}
          wait-on-timeout: 120
          browser: chrome
          record: true
          parallel: true
          group: merge
          install: false
          working-directory: tests/web
          ```

[admah]

@mellis481 yes, I will be providing updates as they’re available.

I will be closing this and updating in #531 since this is a duplicate of that issue.