Lightrun
question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

The same command against the same code results different coverage values

See original GitHub issue

I experience that when coverage data is collected on my build server then certain coverage is missing. The same command on my laptop includes all the coverage data. The result is validated by result of dotCover.

On my laptop the same project with the same command results this (only the important snippet):

| DigitalLibrary.MasterData.Controllers                     | 52.17% | 100%   | 50%    |

On the build server the result is the following (important snippet only)

| DigitalLibrary.MasterData.Controllers                     | 0%     | 100%   | 0%     |

I don’t have any idea what can be the problem here. If any details is needed, please let me know and I’ll do my best to provide it.

Project details

  • project
  • the tests are not collected are integration tests, the spins up a whole webhost to run tests as it is described in the MS doc
  • the test cases are generated from Specflow
  • here are the tests
  • the test subjects are these controllers

Build server

  • added as agent to Azure DevOps

Dotnet info

sayusi@ubi  ~  dotnet --info
.NET SDK (reflecting any global.json):
 Version:   5.0.103
 Commit:    9effbc8ad5

Runtime Environment:
 OS Name:     ubuntu
 OS Version:  20.04
 OS Platform: Linux
 RID:         ubuntu.20.04-x64
 Base Path:   /usr/share/dotnet/sdk/5.0.103/

Host (useful for support):
  Version: 5.0.3
  Commit:  eae88cc11b

.NET SDKs installed:
  3.1.406 [/usr/share/dotnet/sdk]
  5.0.103 [/usr/share/dotnet/sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 3.1.12 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 5.0.3 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 3.1.12 [/usr/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 5.0.3 [/usr/share/dotnet/shared/Microsoft.NETCore.App]

Command

  • the particular build can be found here
2021-02-12T19:05:13.3345252Z [command]/usr/bin/dotnet test /home/sayusi/build_agent/_work/12/s/src/DigitalLibrary.sln --logger trx --results-directory /home/sayusi/build_agent/_work/_temp --no-build --configuration Debug --verbosity normal /p:CollectCoverage=true /p:ExcludeByAttribute='Obsolete*%2cGeneratedCodeAttribute*%2cCompilerGeneratedAttribute' /p:CoverletOutputFormat=opencover /p:CoverletOutput=/home/sayusi/build_agent/_work/_temp/coverage/ /p:MergeWith=/home/sayusi/build_agent/_work/_temp/coverage/

Result

2021-02-12T19:05:52.2136081Z +-----------------------------------------------------------+--------+--------+--------+
2021-02-12T19:05:52.2137238Z | Module                                                    | Line   | Branch | Method |
2021-02-12T19:05:52.2138303Z +-----------------------------------------------------------+--------+--------+--------+
2021-02-12T19:05:52.2139050Z | DigitalLibrary.Utils.MasterDataTestHelper                 | 95.34% | 84.61% | 84.61% |
2021-02-12T19:05:52.2140200Z +-----------------------------------------------------------+--------+--------+--------+
2021-02-12T19:05:52.2140928Z | DigitalLibrary.MasterData.Ctx                             | 100%   | 100%   | 100%   |
2021-02-12T19:05:52.2141967Z +-----------------------------------------------------------+--------+--------+--------+
2021-02-12T19:05:52.2142937Z | DigitalLibrary.ControlPanel.BusinessLogic.Exceptions      | 100%   | 100%   | 100%   |
2021-02-12T19:05:52.2144216Z +-----------------------------------------------------------+--------+--------+--------+
2021-02-12T19:05:52.2145594Z | DigitalLibrary.ControlPanel.WebApi.Api                    | 100%   | 100%   | 100%   |
2021-02-12T19:05:52.2146796Z +-----------------------------------------------------------+--------+--------+--------+
2021-02-12T19:05:52.2147727Z | DigitalLibrary.MasterData.DomainModel.Interfaces          | 100%   | 100%   | 100%   |
2021-02-12T19:05:52.2148712Z +-----------------------------------------------------------+--------+--------+--------+
2021-02-12T19:05:52.2149412Z | DigitalLibrary.Utils.Guards                               | 33.33% | 34.61% | 50%    |
2021-02-12T19:05:52.2150586Z +-----------------------------------------------------------+--------+--------+--------+
2021-02-12T19:05:52.2151313Z | DigitalLibrary.MasterData.BusinessLogic.Implementations   | 37.04% | 16.93% | 42.66% |
2021-02-12T19:05:52.2153020Z +-----------------------------------------------------------+--------+--------+--------+
2021-02-12T19:05:52.2153786Z | DigitalLibrary.MasterData.DomainModel                     | 35.41% | 100%   | 36.95% |
2021-02-12T19:05:52.2154717Z +-----------------------------------------------------------+--------+--------+--------+
2021-02-12T19:05:52.2155615Z | DigitalLibrary.MasterData.BusinessLogic.Exceptions        | 100%   | 100%   | 100%   |
2021-02-12T19:05:52.2156526Z +-----------------------------------------------------------+--------+--------+--------+
2021-02-12T19:05:52.2157581Z | DigitalLibrary.MasterData.Controllers                     | 0%     | 100%   | 0%     |
2021-02-12T19:05:52.2159145Z +-----------------------------------------------------------+--------+--------+--------+
2021-02-12T19:05:52.2160006Z | DigitalLibrary.ControlPanel.Controllers                   | 0%     | 0%     | 0%     |
2021-02-12T19:05:52.2161545Z +-----------------------------------------------------------+--------+--------+--------+
2021-02-12T19:05:52.2162241Z | DigitalLibrary.MasterData.BusinessLogic.ViewModels        | 0%     | 100%   | 0%     |
2021-02-12T19:05:52.2163490Z +-----------------------------------------------------------+--------+--------+--------+
2021-02-12T19:05:52.2164273Z | DigitalLibrary.MasterData.BusinessLogic.Interfaces        | 100%   | 100%   | 100%   |
2021-02-12T19:05:52.2166455Z +-----------------------------------------------------------+--------+--------+--------+
2021-02-12T19:05:52.2168164Z | DigitalLibrary.MasterData.Validators                      | 100%   | 100%   | 100%   |
2021-02-12T19:05:52.2170097Z +-----------------------------------------------------------+--------+--------+--------+
2021-02-12T19:05:52.2171562Z | DigitalLibrary.ControlPanel.BusinessLogic.Interfaces      | 100%   | 100%   | 100%   |
2021-02-12T19:05:52.2173343Z +-----------------------------------------------------------+--------+--------+--------+
2021-02-12T19:05:52.2174731Z | DigitalLibrary.ControlPanel.WebApi.Client                 | 0%     | 0%     | 0%     |
2021-02-12T19:05:52.2176531Z +-----------------------------------------------------------+--------+--------+--------+
2021-02-12T19:05:52.2178471Z | DigitalLibrary.ControlPanel.BusinessLogic.Implementations | 0%     | 0%     | 0%     |
2021-02-12T19:05:52.2180522Z +-----------------------------------------------------------+--------+--------+--------+
2021-02-12T19:05:52.2181871Z | DigitalLibrary.ControlPanel.DomainModel                   | 0%     | 100%   | 0%     |
2021-02-12T19:05:52.2183643Z +-----------------------------------------------------------+--------+--------+--------+
2021-02-12T19:05:52.2184940Z | DigitalLibrary.MasterData.Web.Api                         | 100%   | 100%   | 100%   |
2021-02-12T19:05:52.2186629Z +-----------------------------------------------------------+--------+--------+--------+
2021-02-12T19:05:52.2187957Z | DigitalLibrary.Utils.ControlPanel.DataSample              | 100%   | 100%   | 100%   |
2021-02-12T19:05:52.2189606Z +-----------------------------------------------------------+--------+--------+--------+
2021-02-12T19:05:52.2191290Z | DigitalLibrary.Utils.IntegrationTestFactories             | 0%     | 100%   | 0%     |
2021-02-12T19:05:52.2193724Z +-----------------------------------------------------------+--------+--------+--------+
2021-02-12T19:05:52.2195282Z | DigitalLibrary.ControlPanel.Validators                    | 100%   | 100%   | 100%   |
2021-02-12T19:05:52.2197192Z +-----------------------------------------------------------+--------+--------+--------+
2021-02-12T19:05:52.2198634Z | DigitalLibrary.ControlPanel.Ctx                           | 100%   | 100%   | 100%   |
2021-02-12T19:05:52.2200571Z +-----------------------------------------------------------+--------+--------+--------+
2021-02-12T19:05:52.2202043Z | WebApp                                                    | 0%     | 0%     | 0%     |
2021-02-12T19:05:52.2204001Z +-----------------------------------------------------------+--------+--------+--------+
2021-02-12T19:05:52.2205044Z 
2021-02-12T19:05:52.2206387Z +---------+--------+--------+--------+
2021-02-12T19:05:52.2207471Z |         | Line   | Branch | Method |
2021-02-12T19:05:52.2209037Z +---------+--------+--------+--------+
2021-02-12T19:05:52.2210178Z | Total   | 27.17% | 20.47% | 31.34% |
2021-02-12T19:05:52.2211755Z +---------+--------+--------+--------+
2021-02-12T19:05:52.2212882Z | Average | 58.38% | 76.5%  | 58.92% |
2021-02-12T19:05:52.2214349Z +---------+--------+--------+--------+
2021-02-12T19:05:52.2215311Z

My laptop

  • latest OSX

Dotnet info

andrascsanyi@andras-mbp-2019  DigitalLibrary/src   feature/dimension_structure_tree  dotnet --info
.NET SDK (reflecting any global.json):
 Version:   5.0.102
 Commit:    71365b4d42

Runtime Environment:
 OS Name:     Mac OS X
 OS Version:  11.0
 OS Platform: Darwin
 RID:         osx.11.0-x64
 Base Path:   /usr/local/share/dotnet/sdk/5.0.102/

Host (useful for support):
  Version: 5.0.2
  Commit:  cb5f173b96

.NET SDKs installed:
  5.0.102 [/usr/local/share/dotnet/sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 5.0.2 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 5.0.2 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]

Command

dotnet test --no-build \
        --configuration Debug \
        --verbosity normal \
        /p:CollectCoverage=true \
        /p:ExcludeByAttribute='Obsolete*%2cGeneratedCodeAttribute*%2cCompilerGeneratedAttribute' \
        /p:CoverletOutputFormat=opencover \
        /p:CoverletOutput='/Users/andrascsanyi/Dev/github.com/Andras-Csanyi/DigitalLibrary/src/coverage/' \
        /p:MergeWith='/Users/andrascsanyi/Dev/github.com/Andras-Csanyi/DigitalLibrary/src/coverage/'

Result:

Calculating coverage result...
  Generating report '/Users/andrascsanyi/Dev/github.com/Andras-Csanyi/DigitalLibrary/src/coverage/coverage.opencover.xml'

+-----------------------------------------------------------+--------+--------+--------+
| Module                                                    | Line   | Branch | Method |
+-----------------------------------------------------------+--------+--------+--------+
| DigitalLibrary.ControlPanel.WebApi.Client                 | 0%     | 0%     | 0%     |
+-----------------------------------------------------------+--------+--------+--------+
| DigitalLibrary.Utils.IntegrationTestFactories             | 0%     | 100%   | 0%     |
+-----------------------------------------------------------+--------+--------+--------+
| DigitalLibrary.MasterData.WebApi.Client                   | 100%   | 100%   | 100%   |
+-----------------------------------------------------------+--------+--------+--------+
| DigitalLibrary.MasterData.Controllers                     | 52.17% | 100%   | 50%    |
+-----------------------------------------------------------+--------+--------+--------+
| DigitalLibrary.Utils.MasterDataTestHelper                 | 96.38% | 84.61% | 90%    |
+-----------------------------------------------------------+--------+--------+--------+
| DigitalLibrary.ControlPanel.Validators                    | 100%   | 100%   | 100%   |
+-----------------------------------------------------------+--------+--------+--------+
| DigitalLibrary.ControlPanel.WebApi.Api                    | 100%   | 100%   | 100%   |
+-----------------------------------------------------------+--------+--------+--------+
| DigitalLibrary.Utils.ControlPanel.DataSample              | 100%   | 100%   | 100%   |
+-----------------------------------------------------------+--------+--------+--------+
| DigitalLibrary.ControlPanel.BusinessLogic.Interfaces      | 100%   | 100%   | 100%   |
+-----------------------------------------------------------+--------+--------+--------+
| DigitalLibrary.ControlPanel.BusinessLogic.Exceptions      | 100%   | 100%   | 100%   |
+-----------------------------------------------------------+--------+--------+--------+
| DigitalLibrary.MasterData.BusinessLogic.ViewModels        | 100%   | 100%   | 100%   |
+-----------------------------------------------------------+--------+--------+--------+
| DigitalLibrary.ControlPanel.Ctx                           | 100%   | 100%   | 100%   |
+-----------------------------------------------------------+--------+--------+--------+
| DigitalLibrary.ControlPanel.DomainModel                   | 0%     | 100%   | 0%     |
+-----------------------------------------------------------+--------+--------+--------+
| DigitalLibrary.Utils.Guards                               | 16.66% | 19.23% | 33.33% |
+-----------------------------------------------------------+--------+--------+--------+
| DigitalLibrary.ControlPanel.BusinessLogic.Implementations | 0%     | 0%     | 0%     |
+-----------------------------------------------------------+--------+--------+--------+
| DigitalLibrary.MasterData.DomainModel                     | 33.33% | 100%   | 50%    |
+-----------------------------------------------------------+--------+--------+--------+
| WebApp                                                    | 78.82% | 75%    | 66.66% |
+-----------------------------------------------------------+--------+--------+--------+
| DiLibHttpClientResponseObjects                            | 100%   | 100%   | 100%   |
+-----------------------------------------------------------+--------+--------+--------+
| DigitalLibrary.MasterData.Ctx                             | 100%   | 100%   | 100%   |
+-----------------------------------------------------------+--------+--------+--------+
| DigitalLibrary.Utils.DiLibHttpClient                      | 100%   | 50%    | 100%   |
+-----------------------------------------------------------+--------+--------+--------+
| DigitalLibrary.MasterData.Web.Api.Client.Exceptions       | 100%   | 100%   | 100%   |
+-----------------------------------------------------------+--------+--------+--------+
| DigitalLibrary.MasterData.Web.Api.Client.Interfaces       | 100%   | 100%   | 100%   |
+-----------------------------------------------------------+--------+--------+--------+
| DigitalLibrary.MasterData.BusinessLogic.Implementations   | 100%   | 100%   | 100%   |
+-----------------------------------------------------------+--------+--------+--------+
| DigitalLibrary.MasterData.Validators                      | 100%   | 100%   | 100%   |
+-----------------------------------------------------------+--------+--------+--------+
| DigitalLibrary.MasterData.DomainModel.Interfaces          | 100%   | 100%   | 100%   |
+-----------------------------------------------------------+--------+--------+--------+
| DigitalLibrary.MasterData.BusinessLogic.Interfaces        | 100%   | 100%   | 100%   |
+-----------------------------------------------------------+--------+--------+--------+
| DigitalLibrary.MasterData.BusinessLogic.Exceptions        | 100%   | 100%   | 100%   |
+-----------------------------------------------------------+--------+--------+--------+
| DigitalLibrary.MasterData.Web.Api                         | 100%   | 100%   | 100%   |
+-----------------------------------------------------------+--------+--------+--------+
| DigitalLibrary.ControlPanel.Controllers                   | 0%     | 100%   | 0%     |
+-----------------------------------------------------------+--------+--------+--------+

+---------+--------+--------+--------+
|         | Line   | Branch | Method |
+---------+--------+--------+--------+
| Total   | 78.99% | 44.28% | 69.49% |
+---------+--------+--------+--------+
| Average | 75.08% | 87.2%  | 75.51% |
+---------+--------+--------+--------+

     1>Done Building Project "/Users/andrascsanyi/Dev/github.com/Andras-Csanyi/DigitalLibrary/src/DigitalLibrary.sln" (VSTest target(s)).

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:5

github_iconTop GitHub Comments

1reaction
Andras-Csanyicommented, Feb 15, 2021

@petli Ok, I see it. Thanks for your help. I’ll take a try later today and I’ll let you know.

0reactions
MarcoRossignolicommented, Jan 16, 2022

Closing for stale conversation, feel free to reopen if needed.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Getting different output for same code while comparing two ...
python - Getting different output for same code while comparing two floating point values - Stack Overflow.
Read more >
Code coverage testing - Visual Studio (Windows)
To get fully combined results, use the same build of the code, changing only the test data.
Read more >
What is Code Coverage? | Atlassian
Code coverage is a metric that helps you understand how much of your source is tested. Learn how it is calculated & how...
Read more >
Improve Java Code Coverage and Quality with Unit Tests ...
We're going to use a sample program to explore how code coverage and cyclomatic complexity calculations are useful for making sure that Java ......
Read more >
Everything you need to know about code coverage
Code coverage is a software testing metric that can help in assessing the test performance and quality aspects of any software.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found
Previous page

lcov file is empty when using C# source generator

Next page

Code coverage stops working if assembly contains interfaces from source generators