Assertion failure in async test case does not fail test overall

Related issues

• https://brucelefebvre.com/blog/2019/08/26/cypress-gotchas-with-promises/
• https://github.com/cypress-io/cypress/issues/1417 (this is more about using async/await with the Cypress API, cy.xxx)
• https://github.com/cypress-io/cypress/issues/3724
• https://github.com/cypress-io/cypress/issues/3883
• https://github.com/cypress-io/cypress/issues/3497

Current behavior:

The assertion fails(in red) but the overall test still passes,

Desired behavior:

Asserts with cy work in async test case

Steps to reproduce: (app code and test code)

Use the following testing code with async and notice the test succeeds.

https://github.com/MadLittleMods/cypress-test-tiny/pull/1/files

const generateFixtures = function() {
  return new Promise(resolve => {
    setTimeout(() => {
      resolve("bar");
    }, 1000);
  });
};

describe("Some page", function() {
  it("shows something", async function() {
    const fixtures = await generateFixtures();

    cy.visit(`http://google.com/?foo=${fixtures}`);

    cy.contains("somethingthatodoesNOTexist");
  });
});

Versions

• Cypress 3.4.0
• Node v10.15.1
• Windows 10

[damaon]

What is the point of having testing tool that doesn’t even assert properly? This should have P1

Can someone guide me to related code?

[meissadia]

I believe most of these failures are due to the assertion failing AFTER the test has successfully finished, read https://www.cypress.io/blog/2020/01/16/when-can-the-test-stop/

Very helpful blog post @bahmutov! This approach solved my issue:

it('waits for window confirm to happen using variable', () => {
  cy.visit('index.html')
  let called

  cy.on('window:confirm', (message) => {
    expect(message).to.equal('Are you sure?')
    called = true
  })

  cy.get('#click').click()
  // test automatically waits for the variable "called"
  cy.wrap(null).should(() => {
    expect(called).to.be.true
  })
})