question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Fix intermittent unit/reference test failures

See original GitHub issue

To debug this, pull request #12124 included the browser name in the logs for the unit tests too.

The following is a list of unit test failures we have seen:

  • TEST-UNEXPECTED-FAIL | cleans up document resources during rendering of page | in firefox | Unhandled promise rejection: Error: shall fail cleanup in http://127.0.0.1:38165/node_modules/systemjs/dist/system.js line 4 > eval (line 1585) (Linux; fixed in #12126)
  • TEST-UNEXPECTED-FAIL | creates pdf doc from non-existent URL | in chrome | Unhandled promise rejection: MissingPDFException: Missing PDF "http://127.0.0.1:38165/test/pdfs/non-existent.pdf". (Linux)
  • TEST-UNEXPECTED-FAIL | gets default page mode | in chrome | Unhandled promise rejection: AbortException: Worker was terminated. (Linux; fixed in #12144/#12136)
  • TEST-UNEXPECTED-FAIL | gets non-existent outline | in chrome | Unhandled promise rejection: AbortException: Worker was terminated. (Linux; fixed in #12144/#12136)
  • TEST-UNEXPECTED-FAIL | gets operatorList with JPEG image (issue 4888) | in chrome | Unhandled promise rejection: AbortError: The user aborted a request. (Windows; fixed in #12144/#12136)
  • TEST-UNEXPECTED-FAIL | should correctly render PDFs in parallel | in firefox | Unhandled promise rejection: AbortError: The operation was aborted. (line 369) (Windows; fixed in #12144/#12136)
  • TEST-UNEXPECTED-FAIL | multiple render() on the same canvas | in firefox | Failed: shall fail rendering (Windows; fixed in #12171)

It’s clear from this overview that most problems are related to unexpected action abortions, both in Chrome/Firefox and on Windows/Linux. This wasn’t happening a few days ago, so maybe something regressed? The second one is also interesting since we explicitly catch the exception in the unit test.

Pull request #12125 tries to improve the situation for the reference test failures.

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:21 (17 by maintainers)

github_iconTop GitHub Comments

1reaction
timvandermeijcommented, Mar 7, 2021

Closing since I’ve now opened a new issue with the remaining test failures for a better overview.

1reaction
brendandahlcommented, Nov 6, 2020

For the issue where text on page 11 of tracemonkey disappears I was able to create a “smaller” test case and file a bug with chromium. https://bugs.chromium.org/p/chromium/issues/detail?id=1146296

Read more comments on GitHub >

github_iconTop Results From Across the Web

How to fix intermittent test failures | Sergey Potapov (greyblake)
A blog about software development. How to fix intermittent test failures. Sergey Potapov February 19, 2018 #tests #ci #ruby # ...
Read more >
How to Investigate Intermittent Problems - Satisfice, Inc.
The more testability is designed into a product, the easier it is to investigate and solve intermittent problems. When you have eliminated the...
Read more >
Fixing Intermittent Failing Tests - Collective Idea
Some tricks to help you fix tests that sometimes fail.
Read more >
Bug #1701061 “Intermittent test failure for test_901_remove_unit ...
Hi Frode, there are actually two issues at play here. The amulet error that you've pasted above is a result of a race...
Read more >
How to prevent intermittent test failures - Łukasz Makuch
We've all seen automated tests failing, just to pass correctly after nothing but a restart. ... Well, changing "100 ms" to "200 ms"...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found