Incomplete junit report when having nested requests/tests
See original GitHub issue- Newman Version (can be found via
newman -v
): 3.9.3 - OS details (type, version, and architecture): macOS 10.13.3
- Are you using Newman as a library, or via the CLI? CLI
- Did you encounter this recently, or has this bug always been there: -
- Expected behaviour: The junit reporter should include all assertions in a report (including those of nested requests). The CLI reporter and the postman app are able to show all assertions.
- Command / script used to run Newman:
newman run nestedRequests.postman_collection.json -r cli,junit
- Sample collection, and auxiliary files (minus the sensitive details): https://www.getpostman.com/collections/71794c3e8d7dcc3c1782
- Screenshots (if applicable): -
Steps to reproduce the problem:
- Download & run the collection (see above 7. & 6.)
- Check the junit result stored in
./newman
Details: For a project I have to execute a POST request to queue a job (where I get a UUID as response). Then I have to send a GET request, using the UUID, to retrieve more details on the queued job. I don’t want to copy/paste the same GET request over and over in my collection and therefore I have implemented a nested request within the test script of a folder (and therefore for multiple requests). This works perfectly but when I try to get junit reports via newman, the xml does not include the assertions for nested requests.
The sample collection has two requests:
- The first is a simple request having a
pm.test()
defined as test-script. - The second has a nested request
pm.sendRequest()
and a relatedpm.test()
as test-script.
Issue Analytics
- State:
- Created 6 years ago
- Reactions:2
- Comments:7 (1 by maintainers)
Top Results From Across the Web
Support nested `<testsuite>` (PHPUnit) for JUnit test reports
I adapted the PHPUnit logger to output just one level of <testsuite> and changed the class to classname and it worked like expected....
Read more >JUnit report parsing cannot handle nested suites - Jenkins Jira
Currently the SuiteResult class in Hudson can only handle a single level of nesting, which results in the "None of the test reports...
Read more >JUnit 5 Nested Tests: Grouping Related Tests Together
Learn how to write JUnit 5 nested tests. ... The test has more structure, and related tests are grouped better together.
Read more >JUnit 5 User Guide
Used to configure the test class execution order for @Nested test ... of the assertion methods that JUnit 4 has and adds a...
Read more >Junit Reporter - WebdriverIO
A WebdriverIO reporter that creates Jenkins compatible XML based JUnit reports. ... <testsuite name="a nested test suite" timestamp="2019-04-18T13:45:21" ...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
@gkaiser85 Thanks for reporting this, I was able to reproduce the described condition with the collection you provided. I’ll update this thread when any progress is made. Cheers! 😄
Actually this is exactly the issue. Every request which holds a nested request created with pm.sendRequest fails to output a result. If the nested request is in a folder, every test in the whole folder will end up creating an empty testsuite
<testsuite/>
. To work around that problem we currently have a dedicated request without nesting, but it messes up our collections pretty much, additionally an entry in the list cannot run on its own now, it has to be executed in the right order eg. request1=open session, request2=read data, request3=close session (request1 and 3 should be in the pre-script/test of a folder above request2).