Stuck on an issue?
Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.
And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.
[Feature] Fail suite on retried fail
See original GitHub issueAs for now, there are 2 options available for retries and fast fails
--retries=1 --max-failures=1
If the test is flaky even 1st fail will stop the suite. setting --max-failures=2, will fail on 2 flaky tests failures, which retry would pass.
Could be nice if --max-failures=1 would fail on hard fail, i.e. flaky test failed also in retry
Issue Analytics
- State:
- Created a year ago
- Reactions:2
- Comments:8 (5 by maintainers)
Top Results From Across the Web
Use Suite Retry | Android Open Source Project
The suite retry feature is meant to address those cases: It allows you to retry the failures only instead of the full suites...
Read more >Retry failed test case immediately - Katalon Community
Hi there,. Failed test cases are re-run after test suite execution is ended. If it's retried immediately after failed without waiting for test...
Read more >How to Retry Failed Tests in TestNG Automation Framework
TestNG provides a wonderful feature using which you can retry a test case multiple times before declaring it as Failed. What it means...
Read more >Retrying Failing Tests - JUnit Pioneer
The test failsOnlyOnFirstInvocation is executed once (which fails) and then once more (which succeeds). To allow the entire test suite to pass, the...
Read more >How to retry fails automation tests. | by Anton Smirnov - ITNEXT
The retry feature can enhance this process by minimizing intermittent failures which may occur due to random packet loss.
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
We had a good conversation offline, arriving at setting max-failures to something like 20 - that is a handful of flaky tests that have failed. It would still allow early recovery from the broken infrastructure and would allow us to push retries into the end of the run.
Closing as per above, please feel free to open a new issue if this does not cover your use case.