Calls to CallContext.LogicalSetData made in external libraries during tests error out in VS IDE

From @ekumlin on April 20, 2017 17:55

Description

When calling CallContext.LogicalSetData in an external library during a test method, the VS IDE will return the following error:

An exception occurred while invoking executor ‘executor://mstestadapter/v2’: Unable to find assembly ‘External.Library.Name, Version=2.0.5.0, Culture=neutral, PublicKeyToken=null’.

I expect this issue would happen with calls other than LogicalSetData, but this is the reliable repro we found first.

Notes

• This is a regression from Visual Studio 2015.
• This only happens in the VS IDE. If I run vstest.console.exe on the DLLs, there is no error and the test passes.

Steps to reproduce

(Note: I had some issues getting a .NET Core sample set up. My sample projects all use the net452 framework.)

1. Create a solution with a Class Library (.NET Standard) called External.Library.Name.
2. Add a class called Foo with the following code:

   namespace External.Library.Name
   {
       using System;
       using System.Runtime.Remoting.Messaging;
   
       [Serializable]
       public class Foo
       {
           private static readonly string ValueKey = typeof(Foo).FullName;
   
           public static Foo B()
           {
               return CallContext.LogicalGetData(ValueKey) as Foo ?? new Foo();
           }
   
           public static void A(Foo value)
           {
               CallContext.LogicalSetData(ValueKey, value);
           }
       }
   }
   

3. Build and package the project as a NuGet: msbuild /t:pack External.Library.Name.csproj
4. Move the packaged NuGet to a local NuGet feed (that is referenced by Visual Studio).
5. Create a solution with a Console App (.NET Core) called FooAppX.
6. Add a Class Library (.NET Standard) called Test.FooAppX.
   1. Via the NuGet Package Manager UI, add the appropriate Microsoft.NET.Test.Sdk, MSTest.TestAdapter, and MSTest.TestFramework packages. (We have tried with latest 15.0.0 and 1.1.14 versions.)
   2. Via the NuGet Package Manager UI, add the local External.Library.Name package.
7. Add a test class called Tests with the following code:

   namespace Test.FooAppX
   {
       using External.Library.Name;
       using Microsoft.VisualStudio.TestTools.UnitTesting;
   
       [TestClass]
       public class Tests
       {
           [TestMethod]
           public void Test()
           {
               Foo.A(Foo.B());
           }
       }
   }
   

8. Run the test.

Sample project

ExternalLibraryVsTestExecutorError.zip

Includes:

• FooAppX project
• External.Library.Name project
• External.Library.Name nupkg file

Expected behavior

The test should pass with no errors.

Actual behavior

The test does not run. The error below is shown in the output window.

An exception occurred while invoking executor ‘executor://mstestadapter/v2’: Unable to find assembly ‘External.Library.Name, Version=2.0.5.0, Culture=neutral, PublicKeyToken=null’.

The test will also halt execution; that is, if other tests are available in the app after this test, they will not run either.

Environment

• Windows 10 x64
• Visual Studio 15.1 (26403.7); the bug was first seen in 26403.3

Copied from original issue: Microsoft/vstest#744

[vinishiru]

Same issue here in MSTest V2 - VS 2019 16.6.0.0.

An exception occurred while invoking executor ‘executor://mstestadapter/v2’: Type is not resolved for member ‘log4net.Util.PropertiesDictionary,log4net, Version=2.0.8.0, Culture=neutral, PublicKeyToken=669e0ddf0bb1aa2a’.

I’m using log4net.LogicalThreadContext, which internally calls CallContext.LogicalSetData, so I think it’s the same problem as reported.

Have resolved this using the registering log4net into GAC workaround on ours agent server.

[AbhitejJohn]

Well it actually depends on what mode in the CLI we are comparing this against. If CLI is run only against one test source then yes this is a disparity. However when that is scaled up to multiple test sources then you would find that this scenario fails there. Now IDE by default functions as the later case in CLI because we want to fire up one discovery/execution request for all the test sources(that can be hosted in the same runtime) for performance reasons. That said, this currently also invades into the parent app domain, that isn’t ideal for isolation and could cause in-determinate behavior in a test session for other sources. Would be more problematic in IDE scenarios where a host process can be kept alive as long as that VS instance is alive.