DataProvider gets invoked many times
See original GitHub issueDescription Assume you have a TestNG DataProvider that returns n parameter sets.
If you execute a parameterized Citrus test / DataProvider, the DataProvider method gets invoked n+1 times.
Citrus Version 2.7.7
Expected behavior The DataProvider method should be invoked exactly once at the begin of the test executions.
Actual behavior The DataProvider method gets invoked once at the begin of the test executions and also gets invoked for every of the n parameter sets (resulting in n+1 executions).
Test case sample
public class TestIT extends TestNGCitrusTestRunner {
@DataProvider(name = "dp")
public static Object[][] dataProvider() {
System.out.println("CALLING DATAPROVIDER");
return Stream.of("a", "b", "c").map(s -> new Object[]{s}).toArray(Object[][]::new);
}
@Parameters({"dataProviderValue"})
@CitrusTest
@Test(dataProvider = "dp")
public void testIT(String dataProviderValue) {
// do nothing
}
}
results in the output:
14:19:38,379 DEBUG citrus.Citrus| Loading Citrus application properties
14:19:38,401 DEBUG BeanDefinitionReader| Loaded 0 bean definitions from location pattern [classpath*:citrus-context.xml]
14:19:38,902 INFO port.LoggingReporter|
14:19:38,902 INFO port.LoggingReporter| ------------------------------------------------------------------------
14:19:38,902 INFO port.LoggingReporter| .__ __
14:19:38,902 INFO port.LoggingReporter| ____ |__|/ |________ __ __ ______
14:19:38,902 INFO port.LoggingReporter| _/ ___\| \ __\_ __ \ | \/ ___/
14:19:38,902 INFO port.LoggingReporter| \ \___| || | | | \/ | /\___ \
14:19:38,902 INFO port.LoggingReporter| \___ >__||__| |__| |____//____ >
14:19:38,902 INFO port.LoggingReporter| \/ \/
14:19:38,902 INFO port.LoggingReporter|
14:19:38,902 INFO port.LoggingReporter| C I T R U S T E S T S 2.7.7
14:19:38,902 INFO port.LoggingReporter|
14:19:38,902 INFO port.LoggingReporter| ------------------------------------------------------------------------
14:19:38,903 DEBUG port.LoggingReporter| BEFORE TEST SUITE
14:19:38,903 INFO port.LoggingReporter|
14:19:38,903 INFO port.LoggingReporter|
14:19:38,903 INFO port.LoggingReporter| BEFORE TEST SUITE: SUCCESS
14:19:38,903 INFO port.LoggingReporter| ------------------------------------------------------------------------
14:19:38,903 INFO port.LoggingReporter|
CALLING DATAPROVIDER
14:19:38,919 DEBUG t.TestContextFactory| Created new test context - using global variables: '{}'
CALLING DATAPROVIDER
14:19:38,926 INFO port.LoggingReporter|
14:19:38,926 INFO port.LoggingReporter| ------------------------------------------------------------------------
14:19:38,926 DEBUG port.LoggingReporter| STARTING TEST TestIT.testIT <test.citrus.DataProviderParallel>
14:19:38,926 INFO port.LoggingReporter|
14:19:38,926 DEBUG citrus.TestCase| Initializing test case
14:19:38,927 DEBUG context.TestContext| Setting variable: citrus.test.name with value: 'TestIT.testIT'
14:19:38,927 DEBUG context.TestContext| Setting variable: citrus.test.package with value: 'test.citrus.DataProviderParallel'
14:19:38,927 DEBUG citrus.TestCase| Initializing test parameter 'dataProviderValue' as variable
14:19:38,927 DEBUG context.TestContext| Setting variable: dataProviderValue with value: 'a'
14:19:38,927 DEBUG citrus.TestCase| Test variables:
14:19:38,927 DEBUG citrus.TestCase| citrus.test.name = TestIT.testIT
14:19:38,927 DEBUG citrus.TestCase| dataProviderValue = a
14:19:38,927 DEBUG citrus.TestCase| citrus.test.package = test.citrus.DataProviderParallel
14:19:39,929 INFO port.LoggingReporter|
14:19:39,930 INFO port.LoggingReporter| TEST SUCCESS TestIT.testIT (test.citrus.DataProviderParallel)
14:19:39,930 INFO port.LoggingReporter| ------------------------------------------------------------------------
14:19:39,930 INFO port.LoggingReporter|
14:19:39,931 DEBUG t.TestContextFactory| Created new test context - using global variables: '{}'
CALLING DATAPROVIDER
14:19:39,932 INFO port.LoggingReporter|
14:19:39,932 INFO port.LoggingReporter| ------------------------------------------------------------------------
14:19:39,932 DEBUG port.LoggingReporter| STARTING TEST TestIT.testIT <test.citrus.DataProviderParallel>
14:19:39,932 INFO port.LoggingReporter|
14:19:39,932 DEBUG citrus.TestCase| Initializing test case
14:19:39,932 DEBUG context.TestContext| Setting variable: citrus.test.name with value: 'TestIT.testIT'
14:19:39,932 DEBUG context.TestContext| Setting variable: citrus.test.package with value: 'test.citrus.DataProviderParallel'
14:19:39,932 DEBUG citrus.TestCase| Initializing test parameter 'dataProviderValue' as variable
14:19:39,932 DEBUG context.TestContext| Setting variable: dataProviderValue with value: 'b'
14:19:39,932 DEBUG citrus.TestCase| Test variables:
14:19:39,932 DEBUG citrus.TestCase| citrus.test.name = TestIT.testIT
14:19:39,932 DEBUG citrus.TestCase| dataProviderValue = b
14:19:39,932 DEBUG citrus.TestCase| citrus.test.package = test.citrus.DataProviderParallel
14:19:40,932 INFO port.LoggingReporter|
14:19:40,933 INFO port.LoggingReporter| TEST SUCCESS TestIT.testIT (test.citrus.DataProviderParallel)
14:19:40,933 INFO port.LoggingReporter| ------------------------------------------------------------------------
14:19:40,933 INFO port.LoggingReporter|
14:19:40,934 DEBUG t.TestContextFactory| Created new test context - using global variables: '{}'
CALLING DATAPROVIDER
14:19:40,935 INFO port.LoggingReporter|
14:19:40,935 INFO port.LoggingReporter| ------------------------------------------------------------------------
14:19:40,935 DEBUG port.LoggingReporter| STARTING TEST TestIT.testIT <test.citrus.DataProviderParallel>
14:19:40,935 INFO port.LoggingReporter|
14:19:40,935 DEBUG citrus.TestCase| Initializing test case
14:19:40,935 DEBUG context.TestContext| Setting variable: citrus.test.name with value: 'TestIT.testIT'
14:19:40,935 DEBUG context.TestContext| Setting variable: citrus.test.package with value: 'test.citrus.DataProviderParallel'
14:19:40,935 DEBUG citrus.TestCase| Initializing test parameter 'dataProviderValue' as variable
14:19:40,936 DEBUG context.TestContext| Setting variable: dataProviderValue with value: 'c'
14:19:40,936 DEBUG citrus.TestCase| Test variables:
14:19:40,936 DEBUG citrus.TestCase| citrus.test.name = TestIT.testIT
14:19:40,936 DEBUG citrus.TestCase| dataProviderValue = c
14:19:40,936 DEBUG citrus.TestCase| citrus.test.package = test.citrus.DataProviderParallel
14:19:41,936 INFO port.LoggingReporter|
14:19:41,936 INFO port.LoggingReporter| TEST SUCCESS TestIT.testIT (test.citrus.DataProviderParallel)
14:19:41,936 INFO port.LoggingReporter| ------------------------------------------------------------------------
14:19:41,936 INFO port.LoggingReporter|
14:19:41,940 INFO port.LoggingReporter|
14:19:41,940 INFO port.LoggingReporter| ------------------------------------------------------------------------
14:19:41,940 DEBUG port.LoggingReporter| AFTER TEST SUITE
14:19:41,940 INFO port.LoggingReporter|
14:19:41,940 INFO port.LoggingReporter|
14:19:41,940 INFO port.LoggingReporter| AFTER TEST SUITE: SUCCESS
14:19:41,940 INFO port.LoggingReporter| ------------------------------------------------------------------------
14:19:41,940 INFO port.LoggingReporter|
14:19:41,940 INFO port.LoggingReporter| ------------------------------------------------------------------------
14:19:41,940 INFO port.LoggingReporter|
14:19:41,940 INFO port.LoggingReporter| CITRUS TEST RESULTS
14:19:41,940 INFO port.LoggingReporter|
14:19:41,942 INFO port.LoggingReporter| TestIT.testIT .................................................. SUCCESS
14:19:41,942 INFO port.LoggingReporter| TestIT.testIT .................................................. SUCCESS
14:19:41,942 INFO port.LoggingReporter| TestIT.testIT .................................................. SUCCESS
14:19:41,942 INFO port.LoggingReporter|
14:19:41,942 INFO port.LoggingReporter| TOTAL: 3
14:19:41,942 DEBUG port.LoggingReporter| SKIPPED: 0 (0.0%)
14:19:41,942 INFO port.LoggingReporter| FAILED: 0 (0.0%)
14:19:41,943 INFO port.LoggingReporter| SUCCESS: 3 (100.0%)
14:19:41,943 INFO port.LoggingReporter|
14:19:41,943 INFO port.LoggingReporter| ------------------------------------------------------------------------
14:19:41,943 DEBUG report.HtmlReporter| Generating HTML test report
14:19:41,944 DEBUG util.FileUtils| Reading file resource: 'test-detail.html' (encoding is 'UTF-8')
14:19:41,961 DEBUG util.FileUtils| Reading file resource: 'test-report.html' (encoding is 'UTF-8')
14:19:41,962 INFO ctOutputFileReporter| Generated test report: target/citrus-reports/citrus-test-results.html
14:19:41,963 DEBUG report.JUnitReporter| Generating JUnit test report
14:19:41,964 DEBUG util.FileUtils| Reading file resource: 'junit-test.xml' (encoding is 'UTF-8')
14:19:41,969 DEBUG util.FileUtils| Reading file resource: 'junit-report.xml' (encoding is 'UTF-8')
===============================================
Default Suite
Total tests run: 3, Failures: 0, Skips: 0
===============================================
Process finished with exit code 0
As you can see the line
CALLING DATAPROVIDER
is present in the output 4 times, while it should be present only once.
Note To make sure that this is actually a Citrus issue I created the same test without Citrus / only with TestNG - without Citrus the DataProvider method gets invoked only once (expected behavior).
public class TestWithoutCitrusIT {
@DataProvider(name = "dp")
public static Object[][] dataProvider() {
System.out.println("CALLING DATAPROVIDER");
return Stream.of("a", "b", "c").map(s -> new Object[]{s}).toArray(Object[][]::new);
}
@Parameters({"dataProviderValue"})
@Test(dataProvider = "dp")
public void testIT(String dataProviderValue) {
// print value to make sure that the test gets invoked n times
System.out.println("DataProviderValue " + dataProviderValue);
}
}
results in the output
CALLING DATAPROVIDER
DataProviderValue a
DataProviderValue b
DataProviderValue c
===============================================
Default Suite
Total tests run: 3, Failures: 0, Skips: 0
===============================================
Process finished with exit code 0
As you can see the line
CALLING DATAPROVIDER
is present in the output only once, which is the expected behavior.
Issue Analytics
- State:
- Created 5 years ago
- Comments:6 (3 by maintainers)
@jpeffer contributions are very welcome
Sadly this limitation appears to have existed for years. It seems this should be a higher priority as anyone attempting to use TestNG would not use this in a real-world automation solution.