Feature: Run until test fails (debug flakey/intermittent errors)

Add a way to keep re-running the test-suite until you run into a failure. When debugging flakey, intermittent failures, it would be nice to just keep retrying the tests until something fails. I don’t want to keep pressing re-run manually.

This could be accomplished with cypress run and some outside scripting on top of it but I want to do this inside the cypress open test runner. Using cypress run is also tough to get useful info out of when a failure happens but this is being tracked by https://github.com/cypress-io/cypress/issues/448

There is a Run all tests button in the UI but it only re-runs the test suite once. I want to keep retrying the test suite until some test fails.

---

Related issues but mainly around retrying failed tests,

• https://github.com/cypress-io/cypress/issues/518
• https://github.com/cypress-io/cypress/issues/1599
• https://github.com/cypress-io/cypress/issues/4886

[MadLittleMods]

Thanks @gabbersepp! Created an issue over there, https://github.com/Bkucera/cypress-plugin-retries/issues/37

[alxndr]

I’ve defined a shell function which re-runs the command given until it fails…

until_fail () {
        eval "$@"
        while [ "$?" -eq "0" ]
        do
                eval "$@"
        done
}

Example usage:

$ until_fail npm run -s cypress:run -- --spec cypress/integration/spec_file.js