question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Incomplete junit report when having nested requests/tests

See original GitHub issue
  1. Newman Version (can be found via newman -v): 3.9.3
  2. OS details (type, version, and architecture): macOS 10.13.3
  3. Are you using Newman as a library, or via the CLI? CLI
  4. Did you encounter this recently, or has this bug always been there: -
  5. Expected behaviour: The junit reporter should include all assertions in a report (including those of nested requests). The CLI reporter and the postman app are able to show all assertions.
  6. Command / script used to run Newman: newman run nestedRequests.postman_collection.json -r cli,junit
  7. Sample collection, and auxiliary files (minus the sensitive details): https://www.getpostman.com/collections/71794c3e8d7dcc3c1782
  8. Screenshots (if applicable): -

Steps to reproduce the problem:

  1. Download & run the collection (see above 7. & 6.)
  2. Check the junit result stored in ./newman

Details: For a project I have to execute a POST request to queue a job (where I get a UUID as response). Then I have to send a GET request, using the UUID, to retrieve more details on the queued job. I don’t want to copy/paste the same GET request over and over in my collection and therefore I have implemented a nested request within the test script of a folder (and therefore for multiple requests). This works perfectly but when I try to get junit reports via newman, the xml does not include the assertions for nested requests.

The sample collection has two requests:

  • The first is a simple request having a pm.test() defined as test-script.
  • The second has a nested request pm.sendRequest() and a related pm.test() as test-script.

Issue Analytics

  • State:closed
  • Created 6 years ago
  • Reactions:2
  • Comments:7 (1 by maintainers)

github_iconTop GitHub Comments

4reactions
kunagpalcommented, Apr 16, 2018

@gkaiser85 Thanks for reporting this, I was able to reproduce the described condition with the collection you provided. I’ll update this thread when any progress is made. Cheers! 😄

0reactions
ThisIsABugcommented, Sep 28, 2018

The true is that the issue is not about nested request. The issue could be easily repeated by using pm.sendRequest in Tests - in that case, junit reporter would ignore the result, even if you have pm.test() somewhere in your test definition.

Actually this is exactly the issue. Every request which holds a nested request created with pm.sendRequest fails to output a result. If the nested request is in a folder, every test in the whole folder will end up creating an empty testsuite <testsuite/>. To work around that problem we currently have a dedicated request without nesting, but it messes up our collections pretty much, additionally an entry in the list cannot run on its own now, it has to be executed in the right order eg. request1=open session, request2=read data, request3=close session (request1 and 3 should be in the pre-script/test of a folder above request2).

Read more comments on GitHub >

github_iconTop Results From Across the Web

Support nested `<testsuite>` (PHPUnit) for JUnit test reports
I adapted the PHPUnit logger to output just one level of <testsuite> and changed the class to classname and it worked like expected....
Read more >
JUnit report parsing cannot handle nested suites - Jenkins Jira
Currently the SuiteResult class in Hudson can only handle a single level of nesting, which results in the "None of the test reports...
Read more >
JUnit 5 Nested Tests: Grouping Related Tests Together
Learn how to write JUnit 5 nested tests. ... The test has more structure, and related tests are grouped better together.
Read more >
JUnit 5 User Guide
Used to configure the test class execution order for @Nested test ... of the assertion methods that JUnit 4 has and adds a...
Read more >
Junit Reporter - WebdriverIO
A WebdriverIO reporter that creates Jenkins compatible XML based JUnit reports. ... <testsuite name="a nested test suite" timestamp="2019-04-18T13:45:21" ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found