Building portable DLLs that can be tested

As more of Roslyn moves to target .Net Standard we’re running into a problem with our unit test DLLs. Like our code they need to move to target .Net Standard so we can run our tests on desktop, CoreCLR and across operating systems. Yet there doesn’t seem to be a way to build DLLs which target .Net Standard that are easily tested.

Consider for example the C# syntax unit test project. This targets .Net Standard and runs on CoreCLR / across OS. It does so as a part of our CI system with great success.

This setup though is frustrating because the project can’t be tested directly. The project.json file lacks a runtimes section hence we can’t use <CopyNuGetImplementations>true</CopyNuGetImplementations>. This means the build output of this project lacks the necessary DLLS to make the project runnable. As a result no unit test app works as a developer would expect: test driven.Net, Test Explorer, xunit console runner, etc …

In order to make the tests runnable we have a deployment project: one for desktop and one for CoreCLR. These deployment projects are do-nothing EXE which reference all of our portable unit test assemblies. Because it’s an EXE, the runtimes section is valid, NuGet implementations are copied and the output is runnable. But again none of the standard tooling like TDD.Net / Test Explorer understand this setup. Hence there is always an extra step for developers to take to run their tests.

How can we setup our portable projects to avoid this? This seems like a pretty straight forward problem, hopefully someone has figured out a good solution to this.

We’ve tried adding runtimes sections to the project.json file in our DLL projects. But that only results in loads of NuGet errors.

[jaredpar]

Disagree. This should be an option that I pass to LUT / Test Explorer. Absent of an option they could default to the current runtime.

My project file should not have to change from a standard portable DLL in order to enable tests.

[tmat]

I think this needs to be addressed on multiple layers:

dotnet SDK

Currently one can specify multiple targets for project build via <TargetFrameworks>, which results in the project compiled and deployed multiple times along with all the necessary implementation assemblies and runtime config files for each target framework.

If the project is truly portable (i.e. the same code works on all platforms) repeating the compilation steps is unnecessary. In Roslyn, for example, most tests can be made portable and target netstandard and those that can’t could be moved to separate “Desktop” test projects. We still want to run tests on (possibly multiple versions of) CoreCLR and on Desktop FX.

What I think would be useful here is having the option to specify something like <DeploymentFrameworks> for a project. Setting this property in context of a TargetFramework would result in the compilation outputs compiled for that particular TargetFramework to be copied from the intermediate directory to multiple deployment directories (based on the list in DeploymentFrameworks) along with the runtime assemblies and config files specific to the respective DeploymentFramework. In the Roslyn test scenario, we would specify TargetFramework=netstandard1.3 and DeploymentFrameworks=netcoreapp1.0;netfx461, for example.

vstest/dotnet test/Test Explorer/Live Unit Testing

Ability to select a DeploymentFramework to run tests on. In VS Test Explorer this could be added as a drop-down list to Test Explorer, for example.