Lightrun
question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

DataProvider gets invoked many times

See original GitHub issue

Description Assume you have a TestNG DataProvider that returns n parameter sets.

If you execute a parameterized Citrus test / DataProvider, the DataProvider method gets invoked n+1 times.

Citrus Version 2.7.7

Expected behavior The DataProvider method should be invoked exactly once at the begin of the test executions.

Actual behavior The DataProvider method gets invoked once at the begin of the test executions and also gets invoked for every of the n parameter sets (resulting in n+1 executions).

Test case sample

public class TestIT extends TestNGCitrusTestRunner {

    @DataProvider(name = "dp")
    public static Object[][] dataProvider() {
        System.out.println("CALLING DATAPROVIDER");
        return Stream.of("a", "b", "c").map(s -> new Object[]{s}).toArray(Object[][]::new);
    }

    @Parameters({"dataProviderValue"})
    @CitrusTest
    @Test(dataProvider = "dp")
    public void testIT(String dataProviderValue) {
        // do nothing
    }
}

results in the output:

14:19:38,379 DEBUG        citrus.Citrus| Loading Citrus application properties
14:19:38,401 DEBUG BeanDefinitionReader| Loaded 0 bean definitions from location pattern [classpath*:citrus-context.xml]
14:19:38,902 INFO  port.LoggingReporter| 
14:19:38,902 INFO  port.LoggingReporter| ------------------------------------------------------------------------
14:19:38,902 INFO  port.LoggingReporter|        .__  __                       
14:19:38,902 INFO  port.LoggingReporter|   ____ |__|/  |________ __ __  ______
14:19:38,902 INFO  port.LoggingReporter| _/ ___\|  \   __\_  __ \  |  \/  ___/
14:19:38,902 INFO  port.LoggingReporter| \  \___|  ||  |  |  | \/  |  /\___ \ 
14:19:38,902 INFO  port.LoggingReporter|  \___  >__||__|  |__|  |____//____  >
14:19:38,902 INFO  port.LoggingReporter|      \/                           \/
14:19:38,902 INFO  port.LoggingReporter| 
14:19:38,902 INFO  port.LoggingReporter| C I T R U S  T E S T S  2.7.7
14:19:38,902 INFO  port.LoggingReporter| 
14:19:38,902 INFO  port.LoggingReporter| ------------------------------------------------------------------------
14:19:38,903 DEBUG port.LoggingReporter| BEFORE TEST SUITE
14:19:38,903 INFO  port.LoggingReporter| 
14:19:38,903 INFO  port.LoggingReporter| 
14:19:38,903 INFO  port.LoggingReporter| BEFORE TEST SUITE: SUCCESS
14:19:38,903 INFO  port.LoggingReporter| ------------------------------------------------------------------------
14:19:38,903 INFO  port.LoggingReporter| 
CALLING DATAPROVIDER
14:19:38,919 DEBUG t.TestContextFactory| Created new test context - using global variables: '{}'
CALLING DATAPROVIDER
14:19:38,926 INFO  port.LoggingReporter| 
14:19:38,926 INFO  port.LoggingReporter| ------------------------------------------------------------------------
14:19:38,926 DEBUG port.LoggingReporter| STARTING TEST TestIT.testIT <test.citrus.DataProviderParallel>
14:19:38,926 INFO  port.LoggingReporter| 
14:19:38,926 DEBUG      citrus.TestCase| Initializing test case
14:19:38,927 DEBUG  context.TestContext| Setting variable: citrus.test.name with value: 'TestIT.testIT'
14:19:38,927 DEBUG  context.TestContext| Setting variable: citrus.test.package with value: 'test.citrus.DataProviderParallel'
14:19:38,927 DEBUG      citrus.TestCase| Initializing test parameter 'dataProviderValue' as variable
14:19:38,927 DEBUG  context.TestContext| Setting variable: dataProviderValue with value: 'a'
14:19:38,927 DEBUG      citrus.TestCase| Test variables:
14:19:38,927 DEBUG      citrus.TestCase| citrus.test.name = TestIT.testIT
14:19:38,927 DEBUG      citrus.TestCase| dataProviderValue = a
14:19:38,927 DEBUG      citrus.TestCase| citrus.test.package = test.citrus.DataProviderParallel
14:19:39,929 INFO  port.LoggingReporter| 
14:19:39,930 INFO  port.LoggingReporter| TEST SUCCESS TestIT.testIT (test.citrus.DataProviderParallel)
14:19:39,930 INFO  port.LoggingReporter| ------------------------------------------------------------------------
14:19:39,930 INFO  port.LoggingReporter| 
14:19:39,931 DEBUG t.TestContextFactory| Created new test context - using global variables: '{}'
CALLING DATAPROVIDER
14:19:39,932 INFO  port.LoggingReporter| 
14:19:39,932 INFO  port.LoggingReporter| ------------------------------------------------------------------------
14:19:39,932 DEBUG port.LoggingReporter| STARTING TEST TestIT.testIT <test.citrus.DataProviderParallel>
14:19:39,932 INFO  port.LoggingReporter| 
14:19:39,932 DEBUG      citrus.TestCase| Initializing test case
14:19:39,932 DEBUG  context.TestContext| Setting variable: citrus.test.name with value: 'TestIT.testIT'
14:19:39,932 DEBUG  context.TestContext| Setting variable: citrus.test.package with value: 'test.citrus.DataProviderParallel'
14:19:39,932 DEBUG      citrus.TestCase| Initializing test parameter 'dataProviderValue' as variable
14:19:39,932 DEBUG  context.TestContext| Setting variable: dataProviderValue with value: 'b'
14:19:39,932 DEBUG      citrus.TestCase| Test variables:
14:19:39,932 DEBUG      citrus.TestCase| citrus.test.name = TestIT.testIT
14:19:39,932 DEBUG      citrus.TestCase| dataProviderValue = b
14:19:39,932 DEBUG      citrus.TestCase| citrus.test.package = test.citrus.DataProviderParallel
14:19:40,932 INFO  port.LoggingReporter| 
14:19:40,933 INFO  port.LoggingReporter| TEST SUCCESS TestIT.testIT (test.citrus.DataProviderParallel)
14:19:40,933 INFO  port.LoggingReporter| ------------------------------------------------------------------------
14:19:40,933 INFO  port.LoggingReporter| 
14:19:40,934 DEBUG t.TestContextFactory| Created new test context - using global variables: '{}'
CALLING DATAPROVIDER
14:19:40,935 INFO  port.LoggingReporter| 
14:19:40,935 INFO  port.LoggingReporter| ------------------------------------------------------------------------
14:19:40,935 DEBUG port.LoggingReporter| STARTING TEST TestIT.testIT <test.citrus.DataProviderParallel>
14:19:40,935 INFO  port.LoggingReporter| 
14:19:40,935 DEBUG      citrus.TestCase| Initializing test case
14:19:40,935 DEBUG  context.TestContext| Setting variable: citrus.test.name with value: 'TestIT.testIT'
14:19:40,935 DEBUG  context.TestContext| Setting variable: citrus.test.package with value: 'test.citrus.DataProviderParallel'
14:19:40,935 DEBUG      citrus.TestCase| Initializing test parameter 'dataProviderValue' as variable
14:19:40,936 DEBUG  context.TestContext| Setting variable: dataProviderValue with value: 'c'
14:19:40,936 DEBUG      citrus.TestCase| Test variables:
14:19:40,936 DEBUG      citrus.TestCase| citrus.test.name = TestIT.testIT
14:19:40,936 DEBUG      citrus.TestCase| dataProviderValue = c
14:19:40,936 DEBUG      citrus.TestCase| citrus.test.package = test.citrus.DataProviderParallel
14:19:41,936 INFO  port.LoggingReporter| 
14:19:41,936 INFO  port.LoggingReporter| TEST SUCCESS TestIT.testIT (test.citrus.DataProviderParallel)
14:19:41,936 INFO  port.LoggingReporter| ------------------------------------------------------------------------
14:19:41,936 INFO  port.LoggingReporter| 
14:19:41,940 INFO  port.LoggingReporter| 
14:19:41,940 INFO  port.LoggingReporter| ------------------------------------------------------------------------
14:19:41,940 DEBUG port.LoggingReporter| AFTER TEST SUITE
14:19:41,940 INFO  port.LoggingReporter| 
14:19:41,940 INFO  port.LoggingReporter| 
14:19:41,940 INFO  port.LoggingReporter| AFTER TEST SUITE: SUCCESS
14:19:41,940 INFO  port.LoggingReporter| ------------------------------------------------------------------------
14:19:41,940 INFO  port.LoggingReporter| 
14:19:41,940 INFO  port.LoggingReporter| ------------------------------------------------------------------------
14:19:41,940 INFO  port.LoggingReporter| 
14:19:41,940 INFO  port.LoggingReporter| CITRUS TEST RESULTS
14:19:41,940 INFO  port.LoggingReporter| 
14:19:41,942 INFO  port.LoggingReporter|  TestIT.testIT .................................................. SUCCESS
14:19:41,942 INFO  port.LoggingReporter|  TestIT.testIT .................................................. SUCCESS
14:19:41,942 INFO  port.LoggingReporter|  TestIT.testIT .................................................. SUCCESS
14:19:41,942 INFO  port.LoggingReporter| 
14:19:41,942 INFO  port.LoggingReporter| TOTAL:	3
14:19:41,942 DEBUG port.LoggingReporter| SKIPPED:	0 (0.0%)
14:19:41,942 INFO  port.LoggingReporter| FAILED:	0 (0.0%)
14:19:41,943 INFO  port.LoggingReporter| SUCCESS:	3 (100.0%)
14:19:41,943 INFO  port.LoggingReporter| 
14:19:41,943 INFO  port.LoggingReporter| ------------------------------------------------------------------------
14:19:41,943 DEBUG  report.HtmlReporter| Generating HTML test report
14:19:41,944 DEBUG       util.FileUtils| Reading file resource: 'test-detail.html' (encoding is 'UTF-8')
14:19:41,961 DEBUG       util.FileUtils| Reading file resource: 'test-report.html' (encoding is 'UTF-8')
14:19:41,962 INFO  ctOutputFileReporter| Generated test report: target/citrus-reports/citrus-test-results.html
14:19:41,963 DEBUG report.JUnitReporter| Generating JUnit test report
14:19:41,964 DEBUG       util.FileUtils| Reading file resource: 'junit-test.xml' (encoding is 'UTF-8')
14:19:41,969 DEBUG       util.FileUtils| Reading file resource: 'junit-report.xml' (encoding is 'UTF-8')

===============================================
Default Suite
Total tests run: 3, Failures: 0, Skips: 0
===============================================


Process finished with exit code 0

As you can see the line

CALLING DATAPROVIDER

is present in the output 4 times, while it should be present only once.

Note To make sure that this is actually a Citrus issue I created the same test without Citrus / only with TestNG - without Citrus the DataProvider method gets invoked only once (expected behavior).

public class TestWithoutCitrusIT {

    @DataProvider(name = "dp")
    public static Object[][] dataProvider() {
        System.out.println("CALLING DATAPROVIDER");
        return Stream.of("a", "b", "c").map(s -> new Object[]{s}).toArray(Object[][]::new);
    }

    @Parameters({"dataProviderValue"})
    @Test(dataProvider = "dp")
    public void testIT(String dataProviderValue) {
        // print value to make sure that the test gets invoked n times
        System.out.println("DataProviderValue " + dataProviderValue);
    }
}

results in the output

CALLING DATAPROVIDER
DataProviderValue a
DataProviderValue b
DataProviderValue c

===============================================
Default Suite
Total tests run: 3, Failures: 0, Skips: 0
===============================================


Process finished with exit code 0

As you can see the line

CALLING DATAPROVIDER

is present in the output only once, which is the expected behavior.

Issue Analytics

  • State:open
  • Created 5 years ago
  • Comments:6 (3 by maintainers)

github_iconTop GitHub Comments

1reaction
christophdcommented, Apr 15, 2021

@jpeffer contributions are very welcome

0reactions
jpeffercommented, Apr 15, 2021

Sadly this limitation appears to have existed for years. It seems this should be a higher priority as anyone attempting to use TestNG would not use this in a real-world automation solution.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Data provider being executed multiple times - Stack Overflow
I have problem with using @DataProvider in Arquillian test i'm using so I write a simple test to see how data is being...
Read more >
TestNG @DataProvider for Repeated Tests - HowToDoInJava
The first dimension's size is the number of times the test method will be invoked and the second dimension size contains an array...
Read more >
How To Use DataProviders In TestNG [With Examples]
DataProvider in TestNG is used to inject multiple values into the same test case, this guide explains how to use them in your...
Read more >
Size of DataProvider at runtime - Google Groups
My question: Is it possible to determine at runtime, the number of rows in a dataprovider array i.e., the number of times that...
Read more >
Dataprovider & TestNG XML: Parameterization in Selenium ...
Data provider returns a two-dimensional JAVA object to the test method and the test method, will invoke M times in a M*N type...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found
Previous page

Citrus Build fails with Java 11

Next page

I can not to use value of variable from the Endpointconfig.class and BeforeTest.class