dotnet test --results-directory behavior change

With the 3.1 SDK, the behavior of the --results-directory option for dotnet test changed when running against a solution file.

With the 2.1 SDK, the path specified for the --results-directory was resolved relative to each project directory. With the 3.1 SDK, the path is now resolved relative to the solution directory.

Resolving relative to the solution directory is causing us major headaches. We have a solution with multiple test projects. Our test projects target multiple frameworks, so when we run the tests during CI, we want to collect test results for the matrix [projects x frameworks].

With the 2.1 SDK, we specified the options --framework net461 --results-directory bin/Release/net461, and would get a path like solutionDir/projectDir/bin/Release/net461/TestResults.xml.

With the 3.1 SDK, these options result in the path solutionDir/bin/Release/net461/TestResults.xml, so the result files keep clobbering one another.

Leaving out the --results-directory isn’t really a solution either, since the tests against multiple frameworks end up clobbering one another.

Was this behavior change intentional? Is there a reasonable alternative to customizing the results path on a per-project basis?

[baronfel]

This one missed a label and so the appropriate team wasn’t tagged for follow up. Apologies, folks.

[nohwnd]

I can confirm the change happened, but cannot find the source. I can only assume that this was to make locating a common TestResults directory easier when uploading results. This was enough years ago that rolling this back would not make sense now.

Which logger you are having trouble with? Because TRX will check if there is already a file with the same name and won’t overwrite.

Also when not overriding --results-directory, the files are still put into <project>/TestResults/.