Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Failing to collect code coverage when using .NET Core Fakes preview

See original GitHub issue

Hello guys! I’m facing a problem here probably related to the coverlet.collector package.

I have a .NET Core project for unit tests using the Fakes preview and vstest, but when I try to collect the coverage from the application, an exception is thrown on the method that uses fakes. Can this be an issue on the coverlet library?

output runing the vstest.console Extensions.Net.Http.UnitTests.dll /Collect:"Code Coverage" /Enablecodecoverage /Logger:trx command:

C:\Git\extensions\Extensions.Net.Http.UnitTests\bin\Debug\netcoreapp3.1>vstest.console Extensions.Net.Http.UnitTests.dll /Collect:"Code Coverage" /Enablecodecoverage /Logger:trx
Microsoft (R) Test Execution Command Line Tool Version 16.7.0
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...

A total of 1 test files matched the specified pattern.
Data collector 'UnitTestIsolationV2' message: The data collector 'UnitTestIsolationV2' requested environment variable 'CORECLR_PROFILER' with value '{324F817A-7420-4E6D-B3C1-143FBED6D855}' to be set in test execution environment, but another data collector 'Code Coverage' has already requested same environment variable with different value '{E5F256DC-7959-4DD6-8E4F-C11150AB28E0}'..
Data collector 'UnitTestIsolationV2' message: The data collector 'UnitTestIsolationV2' requested environment variable 'CORECLR_PROFILER_PATH_32' with value 'C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\Common7\IDE\CommonExtensions\Platform\InstrumentationEngine\x86\MicrosoftInstrumentationEngine_x86.dll' to be set in test execution environment, but another data collector 'Code Coverage' has already requested same environment variable with different value 'C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\Common7\IDE\..\..\Team Tools\Dynamic Code Coverage Tools\covrun32.dll'..
Data collector 'UnitTestIsolationV2' message: The data collector 'UnitTestIsolationV2' requested environment variable 'CORECLR_PROFILER_PATH_64' with value 'C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\Common7\IDE\CommonExtensions\Platform\InstrumentationEngine\x64\MicrosoftInstrumentationEngine_x64.dll' to be set in test execution environment, but another data collector 'Code Coverage' has already requested same environment variable with different value 'C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\Common7\IDE\..\..\Team Tools\Dynamic Code Coverage Tools\amd64\covrun64.dll'..
  √ SettingAuthorizationWithNullHttpClientShouldThrowArgumentNullException [15ms]
  √ SettingAuthorizationWithNullAuthenticationHeaderValueShouldThrowArgumentNullException [114ms]
  √ SettingAuthorizationShouldReturnSameHttpClient [7ms]
  √ SettingAuthorizationShouldSetSameAuthorization [1ms]
  √ SettingBaseAddressWithNullHttpClientShouldThrowArgumentNullException [< 1ms]
  √ SettingNullBaseAddressShouldThrowArgumentNullException [< 1ms]
  √ SettingBaseAddressShouldReturnHttpClientWithSameBaseAddress [< 1ms]
  √ SettingBaseAddressShouldReturnSameHttpClient [< 1ms]
  √ SettingTimeoutWithNullHttpClientShouldThrowArgumentNullException [< 1ms]
  √ SettingTimeoutShouldReturnSameHttpClient [< 1ms]
  √ SettingTimeoutShouldReturnHttpClientWithSameTimeout [< 1ms]
  √ SettingInvalidTimeoutShouldThrowArgumentOutOfRangeException [4ms]
  √ SettingInvalidTimeoutShouldThrowArgumentOutOfRangeException (00:00:00) [< 1ms]
  √ SettingInvalidTimeoutShouldThrowArgumentOutOfRangeException (-10675199.02:48:05.4775808) [< 1ms]
  √ SettingInvalidTimeoutShouldThrowArgumentOutOfRangeException (-00:01:00) [< 1ms]
  √ EnsuringSuccessStatusCodeWithErrorWithNullHttpResponseMessageShouldThrowArgumentNullException [6ms]
  √ EnsuringSuccessStatusCodeWithErrorGivenSuccessHttpResponseMessageShouldReturnExpectedHttpResponseMessage [1ms]
  √ EnsuringSuccessStatusCodeWithErrorGivenErrorHttpResponseMessageShouldThrowHttpRequestStatusException [6ms]
  √ EnsuringSuccessStatusCodeWithErrorGivenErrorHttpResponseMessageShouldThrowHttpRequestStatusException (BadRequest,,) [3ms]
  √ EnsuringSuccessStatusCodeWithErrorGivenErrorHttpResponseMessageShouldThrowHttpRequestStatusException (BadRequest,System.Net.Http.StringContent,Content BadRequest) [1ms]
  √ EnsuringSuccessStatusCodeWithErrorGivenErrorHttpResponseMessageShouldThrowHttpRequestStatusException (InternalServerError,System.Net.Http.StringContent,Content InternalServerError) [< 1ms]
  √ EnsuringSuccessStatusCodeWithErrorGivenErrorHttpResponseMessageShouldThrowHttpRequestStatusException (ServiceUnavailable,System.Net.Http.StringContent,Content ServiceUnavailable) [< 1ms]
  √ EnsuringSuccessStatusCodeWithErrorGivenErrorHttpResponseMessageShouldThrowHttpRequestStatusException (Unauthorized,System.Net.Http.StringContent,Content Unauthorized) [< 1ms]
  X ReadingShouldReturnExpectedContent [71ms]
  Error Message:
   Test method Extensions.Net.Http.JsonHttpContentReaderTests.ReadingShouldReturnExpectedContent threw exception:
Microsoft.QualityTools.Testing.Fakes.UnitTestIsolation.UnitTestIsolationException: Failed to get profiler module handle 'C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\Common7\IDE\Extensions\TestPlatform\Extensions\Fakes\x64\Microsoft.QualityTools.Testing.Fakes.Instrumentation.dll'. The specified module could not be found. ---> System.ComponentModel.Win32Exception: The specified module could not be found.
  Stack Trace:
      at Microsoft.QualityTools.Testing.Fakes.UnitTestIsolation.LibraryMethods.GetModuleHandle(String fileName)
   at Microsoft.QualityTools.Testing.Fakes.UnitTestIsolation.IntelliTraceInstrumentationProvider.LoadProfilerModule(String profilerPath)
--- End of inner exception stack trace ---
    at Microsoft.QualityTools.Testing.Fakes.UnitTestIsolation.IntelliTraceInstrumentationProvider.LoadProfilerModule(String profilerPath)
   at Microsoft.QualityTools.Testing.Fakes.UnitTestIsolation.IntelliTraceInstrumentationProvider.Initialize()
   at Microsoft.QualityTools.Testing.Fakes.UnitTestIsolation.UnitTestIsolationRuntime.InitializeUnitTestIsolationInstrumentationProvider()
   at Microsoft.QualityTools.Testing.Fakes.Shims.ShimRuntime.CreateContext()
   at Microsoft.QualityTools.Testing.Fakes.ShimsContext.Create()
   at Extensions.Net.Http.JsonHttpContentReaderTests.ReadingShouldReturnExpectedContent() in C:\Git\extensions\Extensions.Net.Http.UnitTests\JsonHttpContentReaderTests.cs:line 19
   at Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices.ThreadOperations.ExecuteWithAbortSafety(Action action)

  √ ReadingWithNullHttpContentShouldThrowArgumentNullException [9ms]
  √ CreatingContentShouldReturnObjectContentWithCorrectFormatter [22ms]
  √ CreatingContentShouldReturnObjectContentWithCorrectValue [< 1ms]
Results File: C:\Git\extensions\Extensions.Net.Http.UnitTests\bin\Debug\netcoreapp3.1\TestResults\Mateus_Demboski_W101573QV2_2020-09-25_18_29_56.trx

Attachments:
  C:\Git\extensions\Extensions.Net.Http.UnitTests\bin\Debug\netcoreapp3.1\TestResults\7e780545-dcf4-4375-894a-58e7d4942712\Mateus_Demboski_W101573QV2 2020-09-25 18_29_55.coverage
Test Run Failed.
Total tests: 25
     Passed: 24
     Failed: 1
 Total time: 1.8046 Seconds

output running the vstest.console Extensions.Net.Http.UnitTests.dll command:

C:\Git\extensions\Extensions.Net.Http.UnitTests\bin\Debug\netcoreapp3.1>vstest.console Extensions.Net.Http.UnitTests.dll
Microsoft (R) Test Execution Command Line Tool Version 16.7.0
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...

A total of 1 test files matched the specified pattern.
  √ SettingAuthorizationWithNullHttpClientShouldThrowArgumentNullException [12ms]
  √ SettingAuthorizationWithNullAuthenticationHeaderValueShouldThrowArgumentNullException [122ms]
  √ SettingAuthorizationShouldReturnSameHttpClient [5ms]
  √ SettingAuthorizationShouldSetSameAuthorization [1ms]
  √ SettingBaseAddressWithNullHttpClientShouldThrowArgumentNullException [< 1ms]
  √ SettingNullBaseAddressShouldThrowArgumentNullException [< 1ms]
  √ SettingBaseAddressShouldReturnHttpClientWithSameBaseAddress [< 1ms]
  √ SettingBaseAddressShouldReturnSameHttpClient [< 1ms]
  √ SettingTimeoutWithNullHttpClientShouldThrowArgumentNullException [< 1ms]
  √ SettingTimeoutShouldReturnSameHttpClient [< 1ms]
  √ SettingTimeoutShouldReturnHttpClientWithSameTimeout [< 1ms]
  √ SettingInvalidTimeoutShouldThrowArgumentOutOfRangeException [5ms]
  √ SettingInvalidTimeoutShouldThrowArgumentOutOfRangeException (00:00:00) [< 1ms]
  √ SettingInvalidTimeoutShouldThrowArgumentOutOfRangeException (-10675199.02:48:05.4775808) [< 1ms]
  √ SettingInvalidTimeoutShouldThrowArgumentOutOfRangeException (-00:01:00) [< 1ms]
  √ EnsuringSuccessStatusCodeWithErrorWithNullHttpResponseMessageShouldThrowArgumentNullException [6ms]
  √ EnsuringSuccessStatusCodeWithErrorGivenSuccessHttpResponseMessageShouldReturnExpectedHttpResponseMessage [1ms]
  √ EnsuringSuccessStatusCodeWithErrorGivenErrorHttpResponseMessageShouldThrowHttpRequestStatusException [8ms]
  √ EnsuringSuccessStatusCodeWithErrorGivenErrorHttpResponseMessageShouldThrowHttpRequestStatusException (BadRequest,,) [5ms]
  √ EnsuringSuccessStatusCodeWithErrorGivenErrorHttpResponseMessageShouldThrowHttpRequestStatusException (BadRequest,System.Net.Http.StringContent,Content BadRequest) [1ms]
  √ EnsuringSuccessStatusCodeWithErrorGivenErrorHttpResponseMessageShouldThrowHttpRequestStatusException (InternalServerError,System.Net.Http.StringContent,Content InternalServerError) [< 1ms]
  √ EnsuringSuccessStatusCodeWithErrorGivenErrorHttpResponseMessageShouldThrowHttpRequestStatusException (ServiceUnavailable,System.Net.Http.StringContent,Content ServiceUnavailable) [< 1ms]
  √ EnsuringSuccessStatusCodeWithErrorGivenErrorHttpResponseMessageShouldThrowHttpRequestStatusException (Unauthorized,System.Net.Http.StringContent,Content Unauthorized) [< 1ms]
  √ ReadingShouldReturnExpectedContent [50ms]
  √ ReadingWithNullHttpContentShouldThrowArgumentNullException [2ms]
  √ CreatingContentShouldReturnObjectContentWithCorrectFormatter [26ms]
  √ CreatingContentShouldReturnObjectContentWithCorrectValue [< 1ms]

Test Run Successful.
Total tests: 25
     Passed: 25
 Total time: 1.6128 Seconds

about the environment:

Windows 10 Enterprise (1903)
Visual Studio Enterprise 16.7.4

some of the libraries that the project are using:

Microsoft.NET.Test.Sdk 16.7.1
Microsoft.QualityTools.Testing.Fakes 16.7.4-beta.20330.2
MSTest.TestAdapter 2.0.0
MSTest.TestFramework 2.0.0
coverlet.collector 1.3.0

Issue Analytics

State:
Created 3 years ago
Reactions:1
Comments:6 (2 by maintainers)

Top GitHub Comments

1reaction

jakubch1commented, Oct 5, 2020

@mateusdemboski is it running without issues when code coverage is disabled?

Regarding dynamic code coverage (–collect “Code Coverage”) we are building support for .Fakes currently. It should be available in 16.9 release of Visual Studio. You can check demo of this functionality here: https://devblogs.microsoft.com/visualstudio/visual-studio-2019-v16-8-preview-3-1/

1reaction

MarcoRossignolicommented, Oct 3, 2020

Ok let us know if is something related to coverlet…but first glance seems not a coverage issue.