React boundary catches an error but fails the test if test continues

Cypress v4.5.0

Example repo https://github.com/bahmutov/test-custom-error-boundary

npm install
npm run dev

Run the integration spec - one will fail and another is successful

The interesting part: the custom React boundary catches the error, and if use the app manually after the test is finished, it works just fine without throwing

describe('app', () => {
  // but then fails the test
  it('catches an error', () => {
    cy.visit('http://localhost:3000')
    cy.get('.pokemonName-input').type('unknown')
    cy.contains('Submit').click()
    // currently fails
    cy.contains('Try again').should('be.visible')
  })

  it('catches an error and all is good', () => {
    cy.visit('http://localhost:3000')
    cy.get('.pokemonName-input').type('unknown')
    cy.contains('Submit').click()
    // you can click "Try again manually"
  })
})

The second test is happy and no errors leak

[rafahuaman]

I found this description of what React does.

https://github.com/facebook/react/issues/12897#issuecomment-410036991

In development environment, React uses a trick: caught exceptions are thrown inside a fake DOM event which makes them reported by window.onerror, but then React actually catches them so that they don’t propagate up (in case there is an error boundary — otherwise they are rethrown).

In production errors caught by error boundaries stay caught.

[chrisbreiding]

It looks like React still bubbles up the error to window.onerror, which is where Cypress catches it and why it fails the test. You can even see that it displays Cypress’s error because we return that error from window.onerror.

It happens as a result of submitting the unknown pokemon name, but in the second test, it’s too late to trigger failing the test, since it has already passed. It only fails the first test because there are further commands.

There’s room for debate on this, but I think Cypress is doing the right thing. The error hits window.onerror, so it’s natural for Cypress to think that something went wrong. That’s why Cypress gives you a way to turn off failing on uncaught exceptions:

// in test that exercises error boundary
cy.on('uncaught:exception', () => false)

Another option is defining your own window.onerror handler in your app code. Then it overrides Cypress’s handler and, so, won’t fail the test.