Bail option to skip all subsequent tests

Clear and concise description of the problem

Depending on the kind of tests we run, it can be interesting to have a fail-fast feature and exit as soon as a test is failing.

I’m not the only one needing such feature apparently: https://github.com/vitest-dev/vitest/discussions/964

Jest has bail for this purpose.

Suggested solution

As Vitest tries to smooth the transition from Jest, I think the bail option is a sensible name.

But Jest implementation is incomplete/weird, it’s only skipping the next test suites (describe blocks), but it continues to run the other tests within the same test suite.

I think the default should be to bail by test case and not by test suite.

# Skip all the subsequent tests after n test failures (default 1)
--bail <n>

I’m not sure what could be the API if we want to deal with both test cases and test suites, maybe adding a --bail-scope <test|suite> (default test)? 🤔

Alternative

No response

Additional context

Here are related issues about it:

• https://github.com/facebook/jest/issues/8387
• https://github.com/facebook/jest/issues/6527
• https://github.com/facebook/jest/issues/2867

Validations

• Follow our Code of Conduct
• Read the Contributing Guidelines.
• Read the docs.
• Check that there isn’t already an issue that request the same feature to avoid creating a duplicate.

[jtuchel]

May I ask if there is any news?

As already mentioned Jest provides a --bail flag And Playwright provides a --max-failures -x flag

It would save a lot of time in CI pipelines 😃

[jgoux]

1. test failed without describe

test('test 1', () => {/* code that fails */})
test('test 2', () => {/* other test */})

skip test 2 in both test and suite scopes.

1. test failed inside describe

describe('describe 1', () => {
  test('test 1', () => {/* code that fails */})
  test('test 2', () => {/* other test */})
})

describe('describe 2', () => {
  test('test 1', () => {/* other test */})
  test('test 2', () => {/* other test */})
})

skip all the tests in test scope skip only "describe 1"."test 2" in suite scope

1. test failed inside nested describe

describe('describe 1', () => {
  describe('describe 2', () => {
    test('test 1', () => {/* code that fails */})
    test('test 2', () => {/* other test */})
  })

  test('test 1', () => {/* other test */})
  test('test 2', () => {/* other test */})
})

skip all the tests in test scope skip only "describe 2"."test 2" in suite scope

1. test fails in one spec

// spec1.spec.ts
test('test 1', () => {/* code that fails */})
// spec2.spec.ts
test('test 2', () => {/* other test */})

skip all the tests in test scope skip only other tests in spec1.spec.ts in suite scope

I think the rule should be:

• using --bail-scope test, you see the whole tests as a flatten structure, if one test fail, no matter where it fails (in another file, in a describe, whatever) we skip everything else and display the report.
• using --bail-scope suite, you skip the subsequent tests in the same “unit” as the failed test. Keep in mind that this “unit” can be logical, if you have 2 describe blocks named the same in two different test files, they are the same “unit”. Also, it’s more predictable to stop at the closest “unit”, if you have nested describe, I don’t think you want to skip the tests in the parent describe.