JUnit xml output fails with Jenkins xUnit plugin

Our Jenkins uses the xUnit plugin to consume the JUnit xml report. But the plugin detects some validation issues with the generated JUnit report:

WARNING: At line 3 of 18:36:11 WARNING: At line 62 of file:/home/ci-build/workspace/project/report/junit/Chrome%20latest%20(Windows%2010):demo.spec.ts.results.xml:cvc-complex-type.3.2.2: Attribute 'disabled' is not allowed to appear in element 'testsuite'.
ERROR: The result file '/home/ci-build/workspace/project/report/junit/Chrome latest (Windows 10):demo.spec.ts.results.xml' for the metric 'JUnit' is not valid. The result file has been skipped.

I did a test with a JUnit starter project. And the generated JUnit report has a different format as the one form the jasmine-reporters.

jasmine-reporters JUnit report:

<?xml version="1.0" encoding="UTF-8" ?>
<testsuites>
 <testsuite name="Chrome latest (Windows 10):demo.spec.ts.Modal Message Demo" timestamp="2018-06-11T16:33:38" hostname="localhost" time="106.308" errors="0" tests="7" skipped="0" disabled="0" failures="0">
  <testcase classname="Chrome latest (Windows 10):demo.spec.ts.Modal Message Demo" name="should show the open info modal message buttton" time="1.168" />
  <testcase classname="Chrome latest (Windows 10):demo.spec.ts.Modal Message Demo" name="should show the open info modal message buttton without close" time="1.374" />
  <testcase classname="Chrome latest (Windows 10):demo.spec.ts.Modal Message Demo" name="should show the open success modal message buttton" time="1.317" />
  <testcase classname="Chrome latest (Windows 10):demo.spec.ts.Modal Message Demo" name="should show the open warning modal message buttton" time="1.321" />
  <testcase classname="Chrome latest (Windows 10):demo.spec.ts.Modal Message Demo" name="should show the open error modal message buttton" time="1.279" />
  <testcase classname="Chrome latest (Windows 10):demo.spec.ts.Modal Message Demo" name="should show the open custom modal message buttton" time="1.585" />
  <testcase classname="Chrome latest (Windows 10):demo.spec.ts.Modal Message Demo" name="should show the open info modal message buttton closed manually" time="1.302" />
 </testsuite>
 <testsuite name="Chrome latest (Windows 10):demo.spec.ts.Modal Message Demo.Open Info Modal Message" timestamp="2018-06-11T16:33:57" hostname="localhost" time="11.986" errors="0" tests="6" skipped="0" disabled="0" failures="0">
  <testcase classname="Chrome latest (Windows 10):demo.spec.ts.Modal Message Demo.Open Info Modal Message" name="should show the modal message" time="0.956" />
  <testcase classname="Chrome latest (Windows 10):demo.spec.ts.Modal Message Demo.Open Info Modal Message" name="should show the info message icon" time="1.678" />
  <testcase classname="Chrome latest (Windows 10):demo.spec.ts.Modal Message Demo.Open Info Modal Message" name="should show the close buttton" time="1.176" />
  <testcase classname="Chrome latest (Windows 10):demo.spec.ts.Modal Message Demo.Open Info Modal Message" name="should show the cancel buttton" time="1.158" />
  <testcase classname="Chrome latest (Windows 10):demo.spec.ts.Modal Message Demo.Open Info Modal Message" name="should show the action buttton" time="1.2" />
  <testcase classname="Chrome latest (Windows 10):demo.spec.ts.Modal Message Demo.Open Info Modal Message" name="should hide the modal message" time="2.042" />
 </testsuite>
 <testsuite name="Chrome latest (Windows 10):demo.spec.ts.Modal Message Demo.Open Info Modal Message Without Close" timestamp="2018-06-11T16:34:09" hostname="localhost" time="12.108" errors="0" tests="5" skipped="0" disabled="0" failures="0">
  <testcase classname="Chrome latest (Windows 10):demo.spec.ts.Modal Message Demo.Open Info Modal Message Without Close" name="should show the modal message" time="2.209" />
  <testcase classname="Chrome latest (Windows 10):demo.spec.ts.Modal Message Demo.Open Info Modal Message Without Close" name="should show the info message icon" time="1.193" />
  <testcase classname="Chrome latest (Windows 10):demo.spec.ts.Modal Message Demo.Open Info Modal Message Without Close" name="should not show the close buttton" time="0.92" />
  <testcase classname="Chrome latest (Windows 10):demo.spec.ts.Modal Message Demo.Open Info Modal Message Without Close" name="should show the action buttton" time="1.217" />
  <testcase classname="Chrome latest (Windows 10):demo.spec.ts.Modal Message Demo.Open Info Modal Message Without Close" name="should hide the modal message" time="2.774" />
 </testsuite>
 <testsuite name="Chrome latest (Windows 10):demo.spec.ts.Modal Message Demo.Open Success Modal Message" timestamp="2018-06-11T16:34:21" hostname="localhost" time="11.082" errors="0" tests="6" skipped="0" disabled="0" failures="0">
  <testcase classname="Chrome latest (Windows 10):demo.spec.ts.Modal Message Demo.Open Success Modal Message" name="should show the modal message" time="0.925" />
  <testcase classname="Chrome latest (Windows 10):demo.spec.ts.Modal Message Demo.Open Success Modal Message" name="should show the success message icon" time="1.199" />
  <testcase classname="Chrome latest (Windows 10):demo.spec.ts.Modal Message Demo.Open Success Modal Message" name="should show the close buttton" time="1.134" />
  <testcase classname="Chrome latest (Windows 10):demo.spec.ts.Modal Message Demo.Open Success Modal Message" name="should show the cancel buttton" time="1.3" />
  <testcase classname="Chrome latest (Windows 10):demo.spec.ts.Modal Message Demo.Open Success Modal Message" name="should show the action buttton" time="1.137" />
  <testcase classname="Chrome latest (Windows 10):demo.spec.ts.Modal Message Demo.Open Success Modal Message" name="should hide the modal message" time="1.846" />
 </testsuite>
 <testsuite name="Chrome latest (Windows 10):demo.spec.ts.Modal Message Demo.Open Warning Modal Message" timestamp="2018-06-11T16:34:32" hostname="localhost" time="11.378" errors="0" tests="6" skipped="0" disabled="0" failures="0">
  <testcase classname="Chrome latest (Windows 10):demo.spec.ts.Modal Message Demo.Open Warning Modal Message" name="should show the modal message" time="1.07" />
  <testcase classname="Chrome latest (Windows 10):demo.spec.ts.Modal Message Demo.Open Warning Modal Message" name="should show the warning message icon" time="1.197" />
  <testcase classname="Chrome latest (Windows 10):demo.spec.ts.Modal Message Demo.Open Warning Modal Message" name="should show the close buttton" time="1.175" />
  <testcase classname="Chrome latest (Windows 10):demo.spec.ts.Modal Message Demo.Open Warning Modal Message" name="should show the cancel buttton" time="1.199" />
  <testcase classname="Chrome latest (Windows 10):demo.spec.ts.Modal Message Demo.Open Warning Modal Message" name="should show the action buttton" time="1.173" />
  <testcase classname="Chrome latest (Windows 10):demo.spec.ts.Modal Message Demo.Open Warning Modal Message" name="should hide the modal message" time="1.77" />
 </testsuite>
 <testsuite name="Chrome latest (Windows 10):demo.spec.ts.Modal Message Demo.Open Error Modal Message" timestamp="2018-06-11T16:34:43" hostname="localhost" time="12.43" errors="0" tests="6" skipped="0" disabled="0" failures="0">
  <testcase classname="Chrome latest (Windows 10):demo.spec.ts.Modal Message Demo.Open Error Modal Message" name="should show the modal message" time="1.152" />
  <testcase classname="Chrome latest (Windows 10):demo.spec.ts.Modal Message Demo.Open Error Modal Message" name="should show the info message icon" time="1.213" />
  <testcase classname="Chrome latest (Windows 10):demo.spec.ts.Modal Message Demo.Open Error Modal Message" name="should show the close buttton" time="1.263" />
  <testcase classname="Chrome latest (Windows 10):demo.spec.ts.Modal Message Demo.Open Error Modal Message" name="should show the cancel buttton" time="1.141" />
  <testcase classname="Chrome latest (Windows 10):demo.spec.ts.Modal Message Demo.Open Error Modal Message" name="should show the action buttton" time="2.589" />
  <testcase classname="Chrome latest (Windows 10):demo.spec.ts.Modal Message Demo.Open Error Modal Message" name="should hide the modal message" time="1.79" />
 </testsuite>
 <testsuite name="Chrome latest (Windows 10):demo.spec.ts.Modal Message Demo.Open Custom Modal Message" timestamp="2018-06-11T16:34:56" hostname="localhost" time="14.545" errors="0" tests="9" skipped="0" disabled="0" failures="0">
  <testcase classname="Chrome latest (Windows 10):demo.spec.ts.Modal Message Demo.Open Custom Modal Message" name="should show the modal message" time="0.929" />
  <testcase classname="Chrome latest (Windows 10):demo.spec.ts.Modal Message Demo.Open Custom Modal Message" name="should show the day input field" time="1.135" />
  <testcase classname="Chrome latest (Windows 10):demo.spec.ts.Modal Message Demo.Open Custom Modal Message" name="should show the month input field" time="1.169" />
  <testcase classname="Chrome latest (Windows 10):demo.spec.ts.Modal Message Demo.Open Custom Modal Message" name="should show the year input field" time="1.24" />
  <testcase classname="Chrome latest (Windows 10):demo.spec.ts.Modal Message Demo.Open Custom Modal Message" name="should show the confidential icon" time="1.157" />
  <testcase classname="Chrome latest (Windows 10):demo.spec.ts.Modal Message Demo.Open Custom Modal Message" name="should show the close buttton" time="1.15" />
  <testcase classname="Chrome latest (Windows 10):demo.spec.ts.Modal Message Demo.Open Custom Modal Message" name="should show the cancel buttton" time="1.195" />
  <testcase classname="Chrome latest (Windows 10):demo.spec.ts.Modal Message Demo.Open Custom Modal Message" name="should show the action buttton" time="1.141" />
  <testcase classname="Chrome latest (Windows 10):demo.spec.ts.Modal Message Demo.Open Custom Modal Message" name="should hide the modal message" time="1.824" />
 </testsuite>
 <testsuite name="Chrome latest (Windows 10):demo.spec.ts.Modal Message Demo.Open Info Modal Message Closed Manually" timestamp="2018-06-11T16:35:10" hostname="localhost" time="13.915" errors="0" tests="7" skipped="0" disabled="0" failures="0">
  <testcase classname="Chrome latest (Windows 10):demo.spec.ts.Modal Message Demo.Open Info Modal Message Closed Manually" name="should show the modal message" time="0.892" />
  <testcase classname="Chrome latest (Windows 10):demo.spec.ts.Modal Message Demo.Open Info Modal Message Closed Manually" name="should show the info message icon" time="1.184" />
  <testcase classname="Chrome latest (Windows 10):demo.spec.ts.Modal Message Demo.Open Info Modal Message Closed Manually" name="should show the close buttton" time="1.237" />
  <testcase classname="Chrome latest (Windows 10):demo.spec.ts.Modal Message Demo.Open Info Modal Message Closed Manually" name="should show the cancel buttton" time="1.08" />
  <testcase classname="Chrome latest (Windows 10):demo.spec.ts.Modal Message Demo.Open Info Modal Message Closed Manually" name="should show the action buttton" time="1.118" />
  <testcase classname="Chrome latest (Windows 10):demo.spec.ts.Modal Message Demo.Open Info Modal Message Closed Manually" name="should not hide the modal message" time="1.948" />
  <testcase classname="Chrome latest (Windows 10):demo.spec.ts.Modal Message Demo.Open Info Modal Message Closed Manually" name="should hide the modal message" time="1.765" />
 </testsuite>
</testsuites>

JUnit starter report:

<?xml version="1.0" encoding="UTF-8"?>
<testsuite name="com.example.project.CalculatorTests" tests="5" skipped="0" failures="0" errors="0" timestamp="2018-06-12T10:53:04" hostname="hostname" time="0.075">
  <properties/>
  <testcase name="addsTwoNumbers()" classname="com.example.project.CalculatorTests" time="0.008"/>
  <testcase name="add(int, int, int)[1]" classname="com.example.project.CalculatorTests" time="0.019"/>
  <testcase name="add(int, int, int)[2]" classname="com.example.project.CalculatorTests" time="0.001"/>
  <testcase name="add(int, int, int)[3]" classname="com.example.project.CalculatorTests" time="0.001"/>
  <testcase name="add(int, int, int)[4]" classname="com.example.project.CalculatorTests" time="0.001"/>
  <system-out><![CDATA[]]></system-out>
  <system-err><![CDATA[]]></system-err>
</testsuite>

It looks to me that the JUnit format changed and disabled is not allowed anymore, but I could not find any valid documentation for that.

Jenkins: 2.89.4 jasmine-reporters: 2.3.1 xUnit plugin: 2.0.2

[putermancer]

2.3.2 has been published to npm to address this. Add suppressDisabled: true to the options passed to JUnitXMLReporter to prevent disabled=".." from showing up in your XML output.

[putermancer]

The documentation being considered when the change to include disabled 13 months ago was this: http://llg.cubic.org/docs/junit/. As it appears to be causing difficulties for some, I have no problem making it optional to include disabled.

@nochums can you be more specific about what isn’t validating with the nunit reporter? There were some minor changes made to it around Nov 2017, primarily to fix a bug that could cause suites not to be reported correctly in certain focused / ignored suite situations, and minor lint issues which should not have impacted validity. Has the nunit spec evolved and this library is missing changes expected in newer schema versions?

Sorry for the long lead time on this. Maintaining this project is very much an on-the-side thing and things sometimes go months without being addressed, particularly if they are not accompanied by a pull request. In the future, please consider making a pull request to address whatever deficiencies you are encountering or the improvements you want to see. If they’re backwards compatible and not likely to break things for others, e.g. they add or change behavior based on certain options, they’re a lot more likely to get merged quickly.