Jest should avoid printing the individual tests if beforeAll fails

🐛 Bug Report

When beforeAll fails, none of the other tests actually execute, but they all fail with the same reason, and they all print the same error.

Jest should avoid printing the individual tests if beforeAll fails. Right now the implementation just checks if there is an error and if yes, print it and fail the test without actually executing it.

Note that @SimenB asked me to create this https://github.com/facebook/jest/issues/6695#issuecomment-619405847.

To Reproduce

Run a beforeAll() that fails where a couple of tests are executed after. This is the failure received (with the supplied repro below):

 FAIL   src/test.unit.test.js
  test that a 3rd party API remains consistent
    ✕ API function 1
    ✕ API function 2
    ✕ API function 3

  ● test that a 3rd party API remains consistent › API function 1

    expect(received).toBe(expected) // Object.is equality

    Expected: "successful"
    Received: "login"

      1 | describe('test that a 3rd party API remains consistent', () => {
    > 2 |   beforeAll(() => expect('login').toBe('successful')); // this will fail
        |                                   ^
      3 |   test('API function 1', () => expect(1).toBe(1)); // each...
      4 |   test('API function 2', () => expect(2).toBe(2)); // ...of these...
      5 |   test('API function 3', () => expect(3).toBe(3)); // ...will fail too

      at src/test.unit.test.js:2:35

  ● test that a 3rd party API remains consistent › API function 2

    expect(received).toBe(expected) // Object.is equality

    Expected: "successful"
    Received: "login"

      1 | describe('test that a 3rd party API remains consistent', () => {
    > 2 |   beforeAll(() => expect('login').toBe('successful')); // this will fail
        |                                   ^
      3 |   test('API function 1', () => expect(1).toBe(1)); // each...
      4 |   test('API function 2', () => expect(2).toBe(2)); // ...of these...
      5 |   test('API function 3', () => expect(3).toBe(3)); // ...will fail too

      at src/test.unit.test.js:2:35

  ● test that a 3rd party API remains consistent › API function 3

    expect(received).toBe(expected) // Object.is equality

    Expected: "successful"
    Received: "login"

      1 | describe('test that a 3rd party API remains consistent', () => {
    > 2 |   beforeAll(() => expect('login').toBe('successful')); // this will fail
        |                                   ^
      3 |   test('API function 1', () => expect(1).toBe(1)); // each...
      4 |   test('API function 2', () => expect(2).toBe(2)); // ...of these...
      5 |   test('API function 3', () => expect(3).toBe(3)); // ...will fail too

      at src/test.unit.test.js:2:35

Test Suites: 1 failed, 1 total
Tests:       3 failed, 3 total
Snapshots:   0 total
Time:        1.037s

Expected behavior

Jest should avoid printing the individual tests if beforeAll fails. It is highly confusing for users.

Link to repl or repo (highly encouraged)

describe('test that a 3rd party API remains consistent', () => {
  beforeAll(() => expect('login').toBe('successful')); // this will fail
  test('API function 1', () => expect(1).toBe(1)); // each...
  test('API function 2', () => expect(2).toBe(2)); // ...of these...
  test('API function 3', () => expect(3).toBe(3)); // ...will be reported as failed too
});

envinfo

  System:
    OS: Linux 4.15 Ubuntu 18.04.4 LTS (Bionic Beaver)
    CPU: (4) x64 Intel(R) Xeon(R) Gold 6132 CPU @ 2.60GHz
  Binaries:
    Node: 12.15.0 - ~/.config/nvm/12.15.0/bin/node
    npm: 6.13.4 - ~/.config/nvm/12.15.0/bin/npm
  npmPackages:
    jest: 25.1.0 => 25.1.0

[vldmrkl]

@itaizelther thank you 🙏

[itaizelther]

@thernstig I have no write permissions to the existing PR, so I have created a new one: #13273 It is done, you may review it