TRX file cannot be correctly located in VsTest 16.7 in Azure DevOps Pipeline

Description

Starting today, the VsTest 16.7.0 is available in Azure DevOps pipeline. We got an error message at the end of the VsTest step that it clamied TRX was not found when publishing the test. It appears the VsTest was looking for trx file in \agent\_work\_temp\TestResults while it was actually saved to $(Common.TestResultsDirectory)

We were using VsTest 16.6.1 with the same pipeline this past Sunday without problem.

Script

Here is the part of our yml pipeline file:

- task: VisualStudioTestPlatformInstaller@1
  displayName: 'VsTest Platform Installer'
  inputs:
    versionSelector: latestStable

# To avoid vstest execution error, we need to split tests into group by framework version.
- task: VSTest@2
  displayName: 'VsTest - testAssemblies - .Net Framework'
  inputs:
    testSelector: 'testAssemblies'
    testAssemblyVer2: |
      **\MyApp.UnitTest.dll
      !**\obj\**
    searchFolder: '$(System.DefaultWorkingDirectory)'
    vsTestVersion: 'toolsInstaller'
    codeCoverageEnabled: true
    pathtoCustomTestAdapters: '$(System.DefaultWorkingDirectory)/test/UnitTests/MyApp.UnitTest'
    otherConsoleOptions: '/ResultsDirectory:$(Common.TestResultsDirectory)'

Log

Here is the log output:

**************** Starting test execution *********************
C:\agent\_work\_tool\VsTest\16.7.0\x64\tools\net451\Common7\IDE\Extensions\TestPlatform\vstest.console.exe "@C:\agent\_work\_temp\tb3bjofmr20.tmp"
Microsoft (R) Test Execution Command Line Tool Version 16.7.0
Copyright (c) Microsoft Corporation.  All rights reserved.
vstest.console.exe "C:\agent\_work\31\s\test\UnitTests\MyApp.UnitTest\bin\x86\Release\MyApp.UnitTest.dll"
/Settings:"C:\agent\_work\_temp\5k2prh3hdf4.tmp.runsettings"
/Logger:"trx"
/TestAdapterPath:"C:\agent\_work\31\s\test\UnitTests\MyApp.UnitTest"
/ResultsDirectory:C:\agent\_work\31\TestResults
Starting test execution, please wait...
Microsoft (R) Coverage Collection Tool Version 16.0.30319.200


Copyright (c) Microsoft Corporation.  All rights reserved.




Results File: C:\agent\_work\31\TestResults\CMMMM038$_CMMMM038_2020-08-07_14_38_13.trx
Attachments:
  C:\agent\_work\31\TestResults\c52bfe34-15a7-4762-bbff-fbba29cac278\CMMMM038$_CMMMM038 2020-08-07 14_38_06.coverage
Test Run Successful.
Total tests: 5
     Passed: 5
 Total time: 9.0765 Seconds
Vstest.console.exe exited with code 0.
**************** Completed test execution *********************
Test results files: 
##[error]Error occurred while publishing test results : MS.VS.TestService.VstestConsoleAdapter.NoTrxFoundException: No trx files found in the specified test results directory C:\agent\_work\_temp\TestResults.
##[error]Skipping marking test run as complete as test run id 0 is invalid.
Completed TestExecution Model...
##[warning]Vstest failed with error. Check logs for failures. There might be failed tests.
##[error]Error: The process 'C:\agent\_work\_tasks\VSTest_ef087383-ee5e-42c7-9a53-ab56c98420f9\2.170.1\Modules\DTAExecutionHost.exe' failed with exit code 1
##[error]Vstest failed with error. Check logs for failures. There might be failed tests.
Finishing: VsTest - testAssemblies - .Net Framework

Here is the log output for VsTest 16.7.0 installation

==============================================================================
Task         : Visual Studio test platform installer
Description  : Acquire the test platform from nuget.org or the tool cache. Satisfies the ‘vstest’ demand and can be used for running tests and collecting diagnostic data using the Visual Studio Test task.
Version      : 1.151.3
Author       : Microsoft Corporation
Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/tool/vstest-platform-tool-installer
==============================================================================
Starting VsTest platform tools installer task.
==============================================================================
Looking for the latest stable version of the Microsoft.Testplatform.
C:\agent\_work\_tasks\VisualStudioTestPlatformInstaller_2c65196a-54fd-4a02-9be8-d9d1837b7111\1.151.3\nuget.exe list packageid:Microsoft.TestPlatform -NonInteractive -Source https://api.nuget.org/v3/index.json
Microsoft.TestPlatform 16.7.0
C:\agent\_work\_tasks\VisualStudioTestPlatformInstaller_2c65196a-54fd-4a02-9be8-d9d1837b7111\1.151.3\nuget.exe install Microsoft.TestPlatform -Version 16.7.0 -Source https://api.nuget.org/v3/index.json -OutputDirectory C:\agent\_work\_temp\VsTest -NoCache -DirectDownload -NonInteractive
Feeds used:
  https://api.nuget.org/v3/index.json



Attempting to gather dependency information for package 'Microsoft.TestPlatform.16.7.0' with respect to project 'C:\agent\_work\_temp\VsTest', targeting 'Any,Version=v0.0'
Gathering dependency information took 1.26 sec
Attempting to resolve dependencies for package 'Microsoft.TestPlatform.16.7.0' with DependencyBehavior 'Lowest'
Resolving dependency information took 0 ms
Resolving actions to install package 'Microsoft.TestPlatform.16.7.0'
Resolved actions to install package 'Microsoft.TestPlatform.16.7.0'
Retrieving package 'Microsoft.TestPlatform 16.7.0' from 'nuget.org'.
  GET https://api.nuget.org/v3-flatcontainer/microsoft.testplatform/16.7.0/microsoft.testplatform.16.7.0.nupkg
  OK https://api.nuget.org/v3-flatcontainer/microsoft.testplatform/16.7.0/microsoft.testplatform.16.7.0.nupkg 51ms
Adding package 'Microsoft.TestPlatform.16.7.0' to folder 'C:\agent\_work\_temp\VsTest'
Added package 'Microsoft.TestPlatform.16.7.0' to folder 'C:\agent\_work\_temp\VsTest'
Successfully installed 'Microsoft.TestPlatform 16.7.0' to C:\agent\_work\_temp\VsTest
Executing nuget actions took 7.53 sec
Caching tool: VsTest 16.7.0 x64
VsTest will use the Test Platform package found in C:\agent\_work\_tool\VsTest\16.7.0\x64

We were using VsTest 16.6.1 without problem before today.

Environment

The agent is running at Windows 10 1803

[AmadeusW]

FYI our issue turned out to be a deadlocked test. After an hour, the test runner process was ended without producing the .trx file. What we really need here is

1. Azure task to report a message that test runner process was terminated due to timeout
2. The test runner to report that a test (provide a name) did not finish running, so that developer can immediately focus on that test.

[Sanan07]

@aDisplayName I tested using VsTest 16.7.0 with specifying ResultsDirectory and it worked fine.

So, in your case trx file was saved to /ResultsDirectory:C:\agent\_work\31\TestResults as you specify it as argument, but AzDo were looking at it in $(Common.TestResultsDirectory) which has value C:\agent\_work\_temp\TestResults, that why it failed to find it there.

And this MS.VS.TestService.VstestConsoleAdapter.NoTrxFoundException: No trx files found in the specified test results directory looks like a AzDo exception. Maybe @ShreyasRmsft can also help us.

@aDisplayName Could you also provide runsettings file?