Serialization issue with AppDomains and some dlls (e.g. log4net)

Description

Sometimes MSTest fails with ‘Unable to find assembly’ when searching for a nuget package dll

Steps to reproduce

1. Load the project
2. Execute TestMethodLog4Net1

Expected behavior

The project should always be able to find the log4net assembly and load it

Actual behavior

Sometimes MSTest is not able to find the assembly. The problem can be (sometimes) worked around by installed previous versions of MSTest.TestAdapter which will then show the following line:

Multiple versions of same extension found. Selecting the highest version.

But uninstalling the package MSTest.TestAdapter and reinstalling it will bring back the bug.

This is the stacktrace output:

Log level is set to Informational (Default).
Source code repository not available. Some features may not work as expected.
Connected to test environment '< Local Windows Environment >'
Test data store opened in 0,123 sec.
========== Starting test discovery ==========
========== Test discovery skipped: All test containers are up to date ==========
Building Test Projects
========== Starting test run ==========
An exception occurred while invoking executor 'executor://mstestadapter/v2': Unable to find assembly 'log4net, Version=2.0.15.0, Culture=neutral, PublicKeyToken=669e0ddf0bb1aa2a'.
Stack trace:

Server stack trace: 
   at System.Runtime.Serialization.Formatters.Binary.BinaryAssemblyInfo.GetAssembly()
   at System.Runtime.Serialization.Formatters.Binary.ObjectReader.GetType(BinaryAssemblyInfo assemblyInfo, String name)
   at System.Runtime.Serialization.Formatters.Binary.ObjectMap..ctor(String objectName, String[] memberNames, BinaryTypeEnum[] binaryTypeEnumA, Object[] typeInformationA, Int32[] memberAssemIds, ObjectReader objectReader, Int32 objectId, BinaryAssemblyInfo assemblyInfo, SizedArray assemIdToAssemblyTable)
   at System.Runtime.Serialization.Formatters.Binary.__BinaryParser.ReadObjectWithMapTyped(BinaryObjectWithMapTyped record)
   at System.Runtime.Serialization.Formatters.Binary.__BinaryParser.Run()
   at System.Runtime.Serialization.Formatters.Binary.ObjectReader.Deserialize(HeaderHandler handler, __BinaryParser serParser, Boolean fCheck, Boolean isCrossAppDomain, IMethodCallMessage methodCallMessage)
   at System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.Deserialize(Stream serializationStream, HeaderHandler handler, Boolean fCheck, Boolean isCrossAppDomain, IMethodCallMessage methodCallMessage)
   at System.Runtime.Remoting.Channels.CrossAppDomainSerializer.DeserializeObject(MemoryStream stm)
   at System.Runtime.Remoting.Messaging.SmuggledMethodReturnMessage.FixupForNewAppDomain()
   at System.Runtime.Remoting.Channels.CrossAppDomainSink.SyncProcessMessage(IMessage reqMsg)

Exception rethrown at [0]: 
   at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
   at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
   at Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.Execution.UnitTestRunner.RunSingleTest(TestMethod testMethod, IDictionary`2 testContextProperties)
   at Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.Execution.TestExecutionManager.ExecuteTestsWithTestRunner(IEnumerable`1 tests, IRunContext runContext, ITestExecutionRecorder testExecutionRecorder, String source, IDictionary`2 sourceLevelParameters, UnitTestRunner testRunner)
   at Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.Execution.TestExecutionManager.ExecuteTestsInSource(IEnumerable`1 tests, IRunContext runContext, IFrameworkHandle frameworkHandle, String source, Boolean isDeploymentDone)
   at Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.Execution.TestExecutionManager.ExecuteTests(IEnumerable`1 tests, IRunContext runContext, IFrameworkHandle frameworkHandle, Boolean isDeploymentDone)
   at Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.Execution.TestExecutionManager.RunTests(IEnumerable`1 tests, IRunContext runContext, IFrameworkHandle frameworkHandle, TestRunCancellationToken runCancellationToken)
   at Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.MSTestExecutor.RunTests(IEnumerable`1 tests, IRunContext runContext, IFrameworkHandle frameworkHandle)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Execution.RunTestsWithTests.InvokeExecutor(LazyExtension`2 executor, Tuple`2 executorUri, RunContext runContext, IFrameworkHandle frameworkHandle)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Execution.BaseRunTests.<>c__DisplayClass46_0.<RunTestInternalWithExecutors>b__0()
   at Microsoft.VisualStudio.TestPlatform.PlatformAbstractions.PlatformThread.<>c__DisplayClass0_0.<Run>b__0()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Microsoft.VisualStudio.TestPlatform.PlatformAbstractions.PlatformThread.Run(Action action, PlatformApartmentState apartmentState, Boolean waitForCompletion)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Execution.BaseRunTests.TryToRunInStaThread(Action action, Boolean waitForCompletion)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Execution.BaseRunTests.RunTestInternalWithExecutors(IEnumerable`1 executorUriExtensionMap, Int64 totalTests)

========== Test run finished: 0 Tests (0 Passed, 0 Failed, 0 Skipped) run in 1,1 sec ==========

A PoC can be found in https://github.com/elgatov/BugMSTest

Environment

Windows 10 Enterprise N v1809 17763.1935 Test Execution Command Line Tool Version 17.3.0-preview-20220626-01 (x64) Autofac v6.4.0 coverlet.collector v3.1.2 log4net v2.0.15 Microsoft.NET.Test.Sdk v17.3.0 MSTest.TestAdapter v2.2.10 MSTest.TestFramework v2.2.10

AB#1658514

[mike-loux-planview]

Hi @mike-loux-planview, thanks for the bump. I will need to do more investigation to understand exactly what’s causing the issue before being able to provide a fix or workaround as I only did a shallow search so far and directly stumbled upon the fact that the serialization issue could be linked to app domains.

Let me reopen the issue (but I will have to remove the milestone because my bandwidth is a little short).

Thanks, @Evangelink! I understand all too well about short bandwidth. Appreciate you taking the time (however small or long) to look into this some more.

[Evangelink]

Hi @mike-loux-planview, thanks for the bump. I will need to do more investigation to understand exactly what’s causing the issue before being able to provide a fix or workaround as I only did a shallow search so far and directly stumbled upon the fact that the serialization issue could be linked to app domains.

Let me reopen the issue (but I will have to remove the milestone because my bandwidth is a little short).