Test assemblies (containers) are not isolatedd from each other

Description

I’ve update a solution from mstest v1 to mstest v2 (1.4.0) only to get problem with a couple of tests which get problem with assembly loading. All tests run and pass if they are executed one project at a time but two of the projects (OpenRiaServices.DomainsServices.Tools*.Tests) get problem with conflicting assemblies if they are run at the same time as the “*.Client.Test” assembly.

Is there a way for the differet test assemblies/containers to execute so that they do not affect each other (so that assemblies loaded for one test container do not affect other test containers)? (I have tried the Isolated setting without succes).

This can be reproduced both when running vstest.console aswell as from within visual studio.

Source (with azure-pipelines.yml) is availible at https://github.com/Daniel-Svensson/OpenRiaServices/tree/feature/mstest2

I recommend using VS 2015 and RiaServices.sln (follow step in azure-pipelines yml)

Test log is availible here: https://svensson-daniel.visualstudio.com/OpenRiaServices/_build/results?buildId=230&view=ms.vss-test-web.test-result-details

Steps to reproduce

1. Setup azure pipeline with yml support againsts https://github.com/Daniel-Svensson/OpenRiaServices
2. Build the feature/mstest2 branch

or.

1. build RiaServices.sln using VS2015
2. run setup-testdatabases.ps1
3. Start “website full trust” project without debugging
4. Run all tests (client and tooling projects are enough)

Expected behavior

I expect test to pass (different test containers should not affect each other), they should execute in different appdomains (or processes)

Actual behavior

Test fails with message:

System.IO.FileLoadException: API restriction: The assembly 'file:///D:\a\1\s\OpenRiaServices.DomainServices.Client\Test\Desktop\bin\Release\OpenRiaServices.DomainServices.Client.dll' has already loaded from a different location. It cannot be loaded from a new location within the same appdomain.

Environment

azure pipelines hosted build pool .

Operating system, Build version of vstest.console, vs 2015 with all updates (similar behaviour with vstest 15.x)

Package version of MSTest framework and adapter: 1.4.0

[Daniel-Svensson]

@kaadhina Can you at least document that DeploymentItems and using Deployment directories are not supported and will not work for many scenarios where mstest v1 works. Maybe you should also mark it as obsolete.

The code base with tests are a quite large one inherited from microsoft (formerly WCF RIA Services) and I do not have the time to remove them all.

The only current valid workaround is to drop all thoughts about mstestv2 and continue to use v1

[kaadhina]

Please re-open if the above does not help