'dotnet test' in solution folder fails when non-test projects are in the solution

Description

Being able to run ‘dotnet test’ on a solution now is amazing! However, unless that solution contains ONLY test projects it always produces a failure result due to not finding the test sdk. In my experience it is rare for tests to be in a separate solution - they generally don’t exist or are in the same solution as the code they are testing.

It seems logical to me that when running tests on a solution any projects without the test sdk should be ignored. If a project has the test sdk and no tests or is being tested in isolation and does not have the sdk that is likely an error. However if a project does not have the sdk and is being tested as part of a solution that is likely not an error.

Alternatively, if that feels too much like a breaking change then adding an option to filter by patterns on the project name could suffice. Something like dotnet test --projects "*.Tests.*". Since most test projects in my experience have some form of pattern to the name - usually involving the word ‘test’ - this would probably be sufficient for most cases.

Note that the existing --filter <EXPRESSION> switch is insufficient to prevent the exception, presumably because it is processed by the underlying test framework which happens after the missing sdk error is thrown.

Steps to reproduce

‘’’ dotnet new sln -n Solution dotnet new console -n Program dotnet sln Solution.sln add Program/Program.csproj dotnet new mstest -n Tests dotnet sln Solution.sln add Tests/Tests.csproj dotnet test $? ‘’’

Expected behavior

1. Existing tests run.
2. Projects with no test sdk reference are ignored.
3. Return code is zero.

Actual behavior

1. Exiting tests run.
2. Projects with no test sdk reference are run as test projects and fail due to an error.
3. Return code is non-zero. In this case it appears to be 1 but I’m not sure if that is consistent.

Environment data

dotnet --info output: .NET Command Line Tools (2.0.0)

Product Information: Version: 2.0.0 Commit SHA-1 hash: cdcd1928c9

Runtime Environment: OS Name: ubuntu OS Version: 16.04 OS Platform: Linux RID: ubuntu.16.04-x64 Base Path: /usr/share/dotnet/sdk/2.0.0/

Microsoft .NET Core Shared Framework Host

Version : 2.0.0 Build : e8b8861ac7faf042c87a5c2f9f2d04c98b69f28d

What is not clear form this environment output is that I am running in bash via the WSL.

[bencyoung]

We have the same issue. A filter or the ability to point it at a sub-directory would be great

dotnet test tests/

or something like that…

[livarcocc]

This issue was moved to Microsoft/vstest#1129