Test cases are flaking in the FakeOppiaClockTest
See original GitHub issueDescribe the bug
A clear and concise description of what the bug is.
The test cases testGetCurrentCalendar_wallClockMode_returnsCalendarWithCurrentTimeMillis()
and testGetCurrentTimeMs_wallClockMode_returnsCurrentTimeMillis
are flaky.
Failure error -
expected to be in range: (1616173019436..1616173019440)
but was : 1616173019440
at org.oppia.android.testing.time.FakeOppiaClockTest.isWithin(FakeOppiaClockTest.kt:262)
at org.oppia.android.testing.time.FakeOppiaClockTest.testGetCurrentCalendar_wallClockMode_returnsCalendarWithCurrentTimeMillis(FakeOppiaClockTest.kt:194)
Expected behavior The test cases should always pass.
Screenshots If applicable, add screenshots or videos to help explain your problem. When run 100 times, the test case failed 1 time -
Sometimes failing on CI checks -
Issue Analytics
- State:
- Created 3 years ago
- Comments:15 (15 by maintainers)
Top Results From Across the Web
How to Fix Flaky Tests - Semaphore CI
Randomly failing tests are the hardest to debug. Here's a framework you can use to fix them and keep your test suite healthy....
Read more >What are Flaky Tests? | TeamCity CI/CD Guide - JetBrains
Flaky tests are tests that return new results, despite there being no changes to code. Find out why flaky tests matter and how...
Read more >What is a flaky test? Definition from WhatIs.com. - TechTarget
Flaky tests can be caused by various factors: an issue with the newly-written code; an issue with the test itself; some external factor...
Read more >Test Flakiness - Methods for identifying and dealing with flaky ...
A flaky test is a test that both passes and fails periodically without any code changes. Flaky tests are definitely annoying but they...
Read more >How to reduce flaky test failures - CircleCI
If your tests are flaky, they cannot help you find (and fix) all your bugs, which negatively impacts user experience.
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
@Karanjot-singh If you carefully take a look at the assertion of the test -
you can see that we check that if reportedTime lies in the range [ currentTime-2, currentTime+2 ] or not, which is a very small range, although currentTime and reportedTime are initialized one after another but the difference between their actual values can vary more than that. So, the current buffer range for the reportedTime is 4 ms, but we want to increase it to 100 ms. Does that make sense?
@prayutsu It will be helpful if you add the failure result.