Wrong test is run when re-running failed DataProvider tests a second time
See original GitHub issuepublic class DataProviderExample {
@DataProvider(name = "test1")
public Object[][] createData1() {
return new Object[][] {
{ "Cedric", new Integer(36) },
{ "Anne", new Integer(37)},
};
}
//This test method declares that its data should be supplied by the Data Provider named "test1"
@Test(dataProvider = "test1")
public void verifyData1(String n1, Integer n2) {
boolean pass = n1.equals("Cedric");
System.out.println(n1 + " == Cedric - " + pass);
Assert.assertTrue(pass);
}
}
Running the above test results in the following output:
Cedric == Cedric - true
Anne == Cedric - false
===============================================
DataProvider Example Suite
Total tests run: 2, Failures: 1, Skips: 0
===============================================
Running the resulting testng-failed.xml creates the following output:
Anne == Cedric - false
===============================================
Failed suite [DataProvider Example Suite]
Total tests run: 1, Failures: 1, Skips: 0
===============================================
Running the testng-failed.xml created after the previous re-run-failed creates the following output:
Cedric == Cedric - true
===============================================
Failed suite [Failed suite [DataProvider Example Suite]]
Total tests run: 1, Failures: 0, Skips: 0
===============================================
Expected Result:
Second re-run-failed attempts to re-run the test that has been failing (i.e. invocation-number 1, which was added to the testng-failed.xml file after the very first test run), rather than the test that passed in the very first run e.g. resulting in the following output:
Anne == Cedric - false
===============================================
Failed suite [Failed suite [DataProvider Example Suite]]
Total tests run: 1, Failures: 1, Skips: 0
===============================================
Issue Analytics
- State:
- Created 7 years ago
- Comments:5
Top Results From Across the Web
TestNG Retry failing using DataProvider if i include ...
In the example below if i include ITestContext in the parameter then during the retry method Test1() is not called the second time....
Read more >How to use dataProvider and only re-run failed test instances?
I am using dataProvider to pass parameters to a test method. Say dataProvider method passes 10 data sets and it fails 3 times....
Read more >How to Retry Failed Tests in TestNG Automation Framework
In this chapter we will learn How to Retry Failed Tests in TestNG. You must have seen random failure during an automated test...
Read more >Retry Failed Test in TestNG with IRetryAnalyzer
In this article, we will learn how to retry failed test in TestNG with IRetryAnalyzer and also how to rerun all tests from...
Read more >How To Use DataProviders In TestNG [With Examples]
However, TestNG parameters enable us to pass the values only once per execution cycle. To overcome this, we can use DataProvider in TestNG...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
The result is strange:
How it is possible to have 1 failure?
Nice catch, thanks!
I close this one as duplicate and will continue to follow the other one.