[py] CI tests are intermittently failing for pull requests [list of culprit tests]
See original GitHub issueWhat happened?
A fair few of the python tests in CI are a little flaky; resulting in red builds in most python pull requests at the minute; we are merging a few knowing they are ‘flaky’ so let’s get that fixed up.
Edit: There are quite a few; here is a compiled lists of tests that suffer in CI; getting information for them all is pretty time consuming; but they could use some work. Some are harder to recreate than others and they could be due to latency in a CI system; most cases are hitting TimeoutErrors
:
- ~test_full_virtual_authenticator (Todo: Add links)~ (fixed)
- test_click_should_timeout (Todo: Add links)
- test_should_timeout_on_page_load_taking_too_long (Todo: Add links)
- test_collect_log_mutations (BIDI/async) (Todo: Add links)
- test_should_be_able_to_click_element_that_is_out_of_view_in_aframe_that_is_out_of_view[chrome] output
- test_touch_pointer_properties - output
os: ubuntu 20.04
chromedriver: 100.0.4896.75
chrome: 100
Issue Analytics
- State:
- Created a year ago
- Comments:6 (3 by maintainers)
Top Results From Across the Web
[py] CI tests are intermittently failing for pull requests [list of culprit ...
A browser automation framework and ecosystem. Contribute to SeleniumHQ/selenium development by creating an account on GitHub.
Read more >Fixing Flaky Time Based Unit Tests | by EG Tech - Medium
Circle CI Build report showing a single test failing. Though retesting usually solves the issue, these flaky failures increase the amount of time...
Read more >How to auto-reject a pull request if tests are failing (Github ...
The best way to "reject" the pull request is to prevent it from being merged unless the tests pass.
Read more >Finding test coupling - Ned Batchelder
The test failure itself was a UnicodeError while trying to log a warning message involving a username with a non-ASCII character in it....
Read more >7. Regression Testing — VisIt User Manual 3.2.2 documentation
VisIt's test suite involves a combination of python scripts in src/test , raw data in archives in the top-level data directory and data...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
@symonk test_full_virtual_authenticator: (outdated)
hey @symonk can you give some examples of failures in git actions? by looking at the actions of the last few days, i only see
tox-lint
fail inselenium/webdriver/firefox/firefox_binary.py
. thanks 😄