Publish Code Coverage Results: coverage.py Cobertura xml not displaying result

Using Azure DevOps

Issue Description

&view=codecoverage-tab Error message

Code coverage data cannot be displayed. Please verify that a valid directory containing code coverage files, including an index.html was used to publish data.

Task logs

2018-09-20T18:21:24.1776880Z ##[section]Starting: Publish code coverage from $(System.DefaultWorkingDirectory)/**/coverage.xml
2018-09-20T18:21:24.1913796Z ==============================================================================
2018-09-20T18:21:24.1931112Z Task         : Publish Code Coverage Results
2018-09-20T18:21:24.1947780Z Description  : Publish Cobertura or JaCoCo code coverage results from a build
2018-09-20T18:21:24.1965544Z Version      : 1.139.0
2018-09-20T18:21:24.1982934Z Author       : Microsoft Corporation
2018-09-20T18:21:24.2001218Z Help         : [More Information](https://go.microsoft.com/fwlink/?LinkID=626485)
2018-09-20T18:21:24.2019569Z ==============================================================================
2018-09-20T18:21:26.9037822Z Reading code coverage summary from '/home/vsts/work/1/s/coverage.xml'
2018-09-20T18:21:27.0160754Z ##[section]Async Command Start: Publish code coverage
2018-09-20T18:21:27.0179363Z Publishing coverage summary data to TFS server.
2018-09-20T18:21:27.0197668Z  Lines- 2804 of 6777 covered.
2018-09-20T18:21:27.0217477Z  Branches- 0 of 0 covered.
2018-09-20T18:21:27.5243160Z Modifying Cobertura Index file
2018-09-20T18:21:27.5262744Z Publishing code coverage files to TFS server.
2018-09-20T18:21:27.5282332Z Uploading 1 files
2018-09-20T18:21:27.6041098Z File upload succeed.
2018-09-20T18:21:27.6060859Z Published '/tmp/Code Coverage Report_857678' as artifact 'Code Coverage Report_857678'
2018-09-20T18:21:27.6086468Z ##[section]Async Command End: Publish code coverage
2018-09-20T18:21:27.6169492Z ##[section]Finishing: Publish code coverage from $(System.DefaultWorkingDirectory)/**/coverage.xml

Troubleshooting

Lesson learned with other cloud providers: It will be nice if you can provide open-source solutions, so we debug our self the problem, find the error message on GitHub and understand the reason of the message, not like old style black box software.

People are trying different hacks to make the XML compatible with Azure DevOps:

1. https://github.com/dazfuller/PyUnitTestTask/blob/master/src/reportgen.ts
2. https://github.com/Microsoft/vsts-tasks/issues/2186#issuecomment-316074078

Tested both hacks without success, maybe is time to make something that it is working without losing the time of the developers to display coverage results that are working perfectly well in Coveralls.io, CircleCI, etc. more Python-friendly coverage environment 😃

[anishkny]

Ugh. This is unacceptable sorry. Going to have to go with https://coveralls.io/ for now until the dev experience is improved.

[stephanebachelier]

I had the same issue with running code coverage on a Javascript project with nyc. First my configuration in azure-pipelines.yaml was :

- script: |
    ./node_modules/.bin/nyc --reporter=cobertura report
  displayName: 'Build code coverage report'

- task: PublishCodeCoverageResults@1
  inputs:
    codeCoverageTool: 'cobertura'
    summaryFileLocation: '$(System.DefaultWorkingDirectory)/coverage/cobertura-coverage.xml'
    reportDirectory: '$(System.DefaultWorkingDirectory)/coverage'
  displayName: 'Publish code coverage results'

I’ve updated the configuration to provide also an index.html, as I’ve read it should be present to display a summary on the Coverage tab.

Basically I’ve added the --reporter=html to the script that build the code coverage:

- script: |
    ./node_modules/.bin/nyc --reporter=cobertura --reporter=html report
  displayName: 'Build code coverage report'

- task: PublishCodeCoverageResults@1
  inputs:
    codeCoverageTool: 'cobertura'
    summaryFileLocation: '$(System.DefaultWorkingDirectory)/coverage/cobertura-coverage.xml'
    reportDirectory: '$(System.DefaultWorkingDirectory)/coverage'
  displayName: 'Publish code coverage results'

The default output displays the summary but it’s really awful :

I’ve eventually understood that all assets are removed (css, scripts, …). I’ve added a new step to inline all CSS into final index.html file and it looks better !

- script: |
    ./node_modules/.bin/nyc --reporter=cobertura --reporter=html report
    ./node_modules/.bin/juice coverage/index.html coverage/index.html # fix ADO rendering
  displayName: 'Build code coverage report'

- task: PublishCodeCoverageResults@1
  inputs:
    codeCoverageTool: 'cobertura'
    summaryFileLocation: '$(System.DefaultWorkingDirectory)/coverage/cobertura-coverage.xml'
    reportDirectory: '$(System.DefaultWorkingDirectory)/coverage'
  displayName: 'Publish code coverage results'