question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Wrong test is run when re-running failed DataProvider tests a second time

See original GitHub issue
public class DataProviderExample {

    @DataProvider(name = "test1")
    public Object[][] createData1() {
        return new Object[][] {
                { "Cedric", new Integer(36) },
                { "Anne", new Integer(37)},
        };
    }

    //This test method declares that its data should be supplied by the Data Provider named "test1"
    @Test(dataProvider = "test1")
    public void verifyData1(String n1, Integer n2) {
        boolean pass = n1.equals("Cedric");
        System.out.println(n1 + " == Cedric - " + pass);
        Assert.assertTrue(pass);
    }

}

Running the above test results in the following output:

Cedric == Cedric - true
Anne == Cedric - false

===============================================
DataProvider Example Suite
Total tests run: 2, Failures: 1, Skips: 0
===============================================

Running the resulting testng-failed.xml creates the following output:

Anne == Cedric - false

===============================================
Failed suite [DataProvider Example Suite]
Total tests run: 1, Failures: 1, Skips: 0
===============================================

Running the testng-failed.xml created after the previous re-run-failed creates the following output:

Cedric == Cedric - true

===============================================
Failed suite [Failed suite [DataProvider Example Suite]]
Total tests run: 1, Failures: 0, Skips: 0
===============================================

Expected Result:

Second re-run-failed attempts to re-run the test that has been failing (i.e. invocation-number 1, which was added to the testng-failed.xml file after the very first test run), rather than the test that passed in the very first run e.g. resulting in the following output:

Anne == Cedric - false

===============================================
Failed suite [Failed suite [DataProvider Example Suite]]
Total tests run: 1, Failures: 1, Skips: 0
===============================================

Issue Analytics

  • State:closed
  • Created 7 years ago
  • Comments:5

github_iconTop GitHub Comments

1reaction
juherrcommented, Sep 12, 2016

The result is strange:

Cedric == Cedric - true

===============================================
DataProvider Example Suite
Total tests run: 1, Failures: 1, Skips: 0
===============================================

How it is possible to have 1 failure?

0reactions
juherrcommented, Sep 13, 2016

Nice catch, thanks!

I close this one as duplicate and will continue to follow the other one.

Read more comments on GitHub >

github_iconTop Results From Across the Web

TestNG Retry failing using DataProvider if i include ...
In the example below if i include ITestContext in the parameter then during the retry method Test1() is not called the second time....
Read more >
How to use dataProvider and only re-run failed test instances?
I am using dataProvider to pass parameters to a test method. Say dataProvider method passes 10 data sets and it fails 3 times....
Read more >
How to Retry Failed Tests in TestNG Automation Framework
In this chapter we will learn How to Retry Failed Tests in TestNG. You must have seen random failure during an automated test...
Read more >
Retry Failed Test in TestNG with IRetryAnalyzer
In this article, we will learn how to retry failed test in TestNG with IRetryAnalyzer and also how to rerun all tests from...
Read more >
How To Use DataProviders In TestNG [With Examples]
However, TestNG parameters enable us to pass the values only once per execution cycle. To overcome this, we can use DataProvider in TestNG...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found