question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

[Bug] Simple tests fail randomly

See original GitHub issue

Current behavior:

  • Tests passes or
  • Tests fail randomly clicking on menu items due elements not visible?

image

How the test(s) look like:

it('Audiences > Source audiences', function () {
	cy.get('li[id="mainMenu:Audiences"]').should('be.visible').click()
	cy.get('li[id="mainMenu:SourceAudiences"]').should('be.visible').click()
        cy.url().should('eq', 'https://domain.com/audiences')
	cy.get('h1[id="screen-title"]').should('have.text', 'Your audiences')
	cy.get('h4[id="screen-subtitle"]').should('have.text', 'Manage & create your source audiences')
})

or even when i run the test like this the same bevavior

it('Audiences > Source audiences', function () {
	cy.get('li[id="mainMenu:Audiences"]').click()
	cy.get('li[id="mainMenu:SourceAudiences"]').click()
        cy.url().should('eq', 'https://domain.com/audiences')
	cy.get('h1[id="screen-title"]').should('have.text', 'Your audiences')
	cy.get('h4[id="screen-subtitle"]').should('have.text', 'Manage & create your source audiences')
})

Also happening when running headless mode.

Desired behavior:

Tests should always pass since the snapshots are showing the elements just fine? Sometimes the elements seems not to be recognized as visible in cypress GUI.

Versions

Cypress: 3.0.0 / Windows 10

Issue Analytics

  • State:closed
  • Created 5 years ago
  • Comments:6 (2 by maintainers)

github_iconTop GitHub Comments

6reactions
pgroot91commented, Apr 25, 2019

I solved this by adding cy.wait() between the navigation. It takes around +/- 3 times longer for the full test to run, but at-least the tests are all passing now! The cause of the test randomly failing seems to be the loading time of iframe(s) on those pages. When the iframe is not completely loaded test(s) fails.

4reactions
jennifer-shehanecommented, Apr 25, 2019

Unfortunately we have to close this issue as there is not enough information to reproduce the problem.

Please comment in this issue with a reproducible example and we will reopen the issue. 🙏

Read more comments on GitHub >

github_iconTop Results From Across the Web

Why The Tests Are Randomly Failing - Building Better Teams
There are three general reasons that “random” is causing tests to fail: (i) the randomly generated data is showing real errors in the...
Read more >
"Randomly" failing tests | Apple Developer Forums
We have a test suite that passes when tests are run manually via Xcode. But when run through Xcode Server on about 50...
Read more >
How to Fix Flaky Tests - Semaphore CI
Randomly failing tests are the hardest to debug. Here's a framework you can use to fix them and keep your test suite healthy....
Read more >
Make Your Tests Fail Randomly (and Profit) - Riccardo Odone
Using random values in automated tests to improve readability and the likelihood of failures in the presence of bugs or wrong mental models....
Read more >
Debugging Intermittent Test Failures - Firefox Source Docs
Intermittents are test failures which happen intermittently, in a seemingly random way. Often you'll write a test that passes fine locally on your...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found