Zero lines covered in Cobertura yields confusing error messaage

Required Information

• Type: Bug
• Task Name: PublishCodeCoverageResults

Environment

• Server - Azure Pipelines
• Agent - Hosted / Ubuntu

Issue Description

I’m setting up a build for a brand new .NET Core project. There are skeleton projects in there that have not been totally filled out yet, but it’s enough to compile and run Hello World. As part of that, I’m setting up coverage reporting using Coverlet and Cobertura output.

When I run tests with coverage, I do get a valid XML coverage report coming out. I then run that through the report generator extension and, finally, use the aggregated report to PublishCodeCoverageResults.

However, while there is a report there is not currently any actual line coverage because the tests aren’t quite there yet. This yields a very confusing error message withe a debug log like this:

##[debug]findPath: '/home/vsts/work/1/a/log/coverage/Cobertura.xml'
##[debug]statOnly: 'true'
##[debug]found 1 paths
##[debug]applying include pattern
##[debug]1 matches
##[debug]1 final results
##[debug]Found 1 result(s) matching pattern: /home/vsts/work/1/a/log/coverage/Cobertura.xml
##[debug]Resolved summary file: /home/vsts/work/1/a/log/coverage/Cobertura.xml
##[debug]Reading XML file: /home/vsts/work/1/a/log/coverage/Cobertura.xml
##[debug]Converting XML file to JSON
##[debug]task result: Failed
##[error]No code coverage results were found to publish.

There are coverage results and they could be published but it’s saying nothing was found.

The error is here:

else if (codeCoverageTool.toLowerCase() === 'cobertura' && resp.coverage) {
  let lines_covered: number = Number(resp.coverage.$["lines-covered"]);
  if (lines_covered && lines_covered > 0) {
    return Q.resolve(false);
  }
}

Basically, it says if there are zero lines covered, return that there was no coverage found. Way up the stack, that yields a very generic “no results” message. The message shown in the logs says that there weren’t any results; not that the results showed no coverage.

While this seems like a minor semantic issue, I honestly just sank like a day into debugging deep into the task trying to figure out whether there was a problem with the report format, or the report generator plugin, or whether maybe the xml2js module couldn’t read the file, or something else. It might be good to have a debug message in there to say something about how there are results it’s just that there’s nothing of value to report. It’d have saved a lot of time.

[tillig]

The error message here still needs improvement.

[github-actions[bot]]

This issue is stale because it has been open for 180 days with no activity. Remove the stale label or comment on the issue otherwise this will be closed in 5 days