Expect is failing but test is passing in cy.on confirm & alert message

I wanted to test my content of the confirmation message but it seems that even if my test fails my case passes

it('Should Get Confirm Message', () => {
        cy.get('.button')
            .click();
        cy.on('window:confirm', str => {
            expect(2 + 2).to.equal(5);
        })
    })

The issue seems to be similar to https://github.com/cypress-io/cypress/issues/3497 and https://github.com/cypress-io/cypress/issues/2413

But their solution does not work for me. The issue persists for window:alert

[dwelle]

Here’s my take without using state:

cy.wrap(new Promise((resolve, reject) => {
  cy.on('window:confirm', msg => {
    try {
      // expect(msg).to.eq(confirmMessage); // optional
    } catch ( err ) {
      return reject(err);
    }
    resolve();
  });
  // set a timeout to ensure we don't wait forever
  setTimeout(() => {
    reject(new Error('window.confirm wasn\'t called within 4s'));
  }, 4000);
}), { log: false });

[bahmutov]

Here is an example waiting for the confirm to have been called before finishing the test (the code calls window.confirm 1 second after the click), see https://github.com/cypress-io/cypress-test-tiny/tree/confirm-after-test

Without waiting

/// <reference types="cypress" />
it('confirms', () => {
  cy.visit('index.html')

  cy.on('window:confirm', (text) => {
    // make the assertion fail on purpose
    // but the test does NOT wait for this assertion to pass
    // By the time the assertion fails, the test has passed
    expect(false).to.be.true
  })

  cy.get('#click').click()
})

With waiting

/// <reference types="cypress" />
it('confirms', () => {
  cy.visit('index.html')

  let called
  cy.on('window:confirm', (text) => {
    // make the assertion fail on purpose
    // but the test does NOT wait for this assertion to pass
    // By the time the assertion fails, the test has passed
    expect(false).to.be.true
    called = true
  })

  cy.get('#click').click()
  cy.should(() => {
    expect(called).to.be.true
  })
})

When the test passes (comment out the false assertion, or change it to pass), then all is good

When the assertion is false, the test correctly fails (even if the error message claims the assertion was thrown from the application code)

I think all assertions that pass or fail AFTER the test has finished should have a warning icon ⚠️ next to them in the Command Log. On click, the DevTools should print a message in the console, explaining

This assertion failed / passed AFTER the test has completed.
Please ensure the test waits for every assertion before completion.
See `https://on.cypress.io/assertion-after-test-finished`