Fix intermittent unit/reference test failures
See original GitHub issueTo debug this, pull request #12124 included the browser name in the logs for the unit tests too.
The following is a list of unit test failures we have seen:
-
TEST-UNEXPECTED-FAIL | cleans up document resources during rendering of page | in firefox | Unhandled promise rejection: Error: shall fail cleanup in http://127.0.0.1:38165/node_modules/systemjs/dist/system.js line 4 > eval (line 1585)
(Linux; fixed in #12126) -
TEST-UNEXPECTED-FAIL | creates pdf doc from non-existent URL | in chrome | Unhandled promise rejection: MissingPDFException: Missing PDF "http://127.0.0.1:38165/test/pdfs/non-existent.pdf".
(Linux) -
TEST-UNEXPECTED-FAIL | gets default page mode | in chrome | Unhandled promise rejection: AbortException: Worker was terminated.
(Linux; fixed in #12144/#12136) -
TEST-UNEXPECTED-FAIL | gets non-existent outline | in chrome | Unhandled promise rejection: AbortException: Worker was terminated.
(Linux; fixed in #12144/#12136) -
TEST-UNEXPECTED-FAIL | gets operatorList with JPEG image (issue 4888) | in chrome | Unhandled promise rejection: AbortError: The user aborted a request.
(Windows; fixed in #12144/#12136) -
TEST-UNEXPECTED-FAIL | should correctly render PDFs in parallel | in firefox | Unhandled promise rejection: AbortError: The operation was aborted. (line 369)
(Windows; fixed in #12144/#12136) -
TEST-UNEXPECTED-FAIL | multiple render() on the same canvas | in firefox | Failed: shall fail rendering
(Windows; fixed in #12171)
It’s clear from this overview that most problems are related to unexpected action abortions, both in Chrome/Firefox and on Windows/Linux. This wasn’t happening a few days ago, so maybe something regressed? The second one is also interesting since we explicitly catch the exception in the unit test.
Pull request #12125 tries to improve the situation for the reference test failures.
Issue Analytics
- State:
- Created 3 years ago
- Comments:21 (17 by maintainers)
Top Results From Across the Web
How to fix intermittent test failures | Sergey Potapov (greyblake)
A blog about software development. How to fix intermittent test failures. Sergey Potapov February 19, 2018 #tests #ci #ruby # ...
Read more >How to Investigate Intermittent Problems - Satisfice, Inc.
The more testability is designed into a product, the easier it is to investigate and solve intermittent problems. When you have eliminated the...
Read more >Fixing Intermittent Failing Tests - Collective Idea
Some tricks to help you fix tests that sometimes fail.
Read more >Bug #1701061 “Intermittent test failure for test_901_remove_unit ...
Hi Frode, there are actually two issues at play here. The amulet error that you've pasted above is a result of a race...
Read more >How to prevent intermittent test failures - Łukasz Makuch
We've all seen automated tests failing, just to pass correctly after nothing but a restart. ... Well, changing "100 ms" to "200 ms"...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
Closing since I’ve now opened a new issue with the remaining test failures for a better overview.
For the issue where text on page 11 of tracemonkey disappears I was able to create a “smaller” test case and file a bug with chromium. https://bugs.chromium.org/p/chromium/issues/detail?id=1146296