question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Test categories aren't propagated to vstest trx logs

See original GitHub issue

With the latest version of the nuni3-adapter unit test descriptions in trx files created with vstest.console (2017) don’t have TestCategory elements.

Expected behaviour

All categories assigned to test cases are reflected in TestCategory elements in trx logs, for example:

    <UnitTest name="ThatEmptyContainsHeaders" storage="d:\work\nunit3-vs-adapter\src\nunittestadaptertests\bin\debug\netcoreapp1.0\nunit.visualstudio.testadapter.tests.dll" id="5d22f9c8-13f7-4be4-b020-438dbf715a39">
      <TestCategory>
        <TestCategoryItem TestCategory="DumpXml" />
      </TestCategory>
      <Execution id="cf9ffa57-0013-4c9a-877e-5104b828b8ec" />
      <TestMethod codeBase="D:\work\nunit3-vs-adapter\src\NUnitTestAdapterTests\bin\Debug\netcoreapp1.0\NUnit.VisualStudio.TestAdapter.Tests.dll" executorUriOfAdapter="executor://nunit3testexecutor/" className="NUnit.VisualStudio.TestAdapter.Tests.DumpXmlTests" name="ThatEmptyContainsHeaders" />
    </UnitTest>

Repro steps

  1. build the nunit3-vs-adapter repo;
  2. invoke vstest.console with the following arguments. note the logger argument
"C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe" "D:\work\nunit3-vs-adapter\src\NUnitTestAdapterTests\bin\Debug\netcoreapp1.0\NUnit.VisualStudio.TestAdapter.Tests.dll" /logger:"trx" /TestAdapterPath:"D:\work\nunit3-vs-adapter\bin\Debug"
  1. the test will spit out a new result trx-file. Open it in a text editor and note that none of the test cases have TestCategory items, for example:
    <UnitTest name="ThatEmptyContainsHeaders" storage="d:\work\nunit3-vs-adapter\src\nunittestadaptertests\bin\debug\netcoreapp1.0\nunit.visualstudio.testadapter.tests.dll" id="c3d0232c-20e9-44be-a6e9-c48b0b4ef694">
      <Execution id="7a1b8f5c-23fd-4562-832f-3febbda631f9" />
      <TestMethod codeBase="D:\work\nunit3-vs-adapter\src\NUnitTestAdapterTests\bin\Debug\netcoreapp1.0\NUnit.VisualStudio.TestAdapter.Tests.dll" executorUriOfAdapter="executor://nunit3testexecutor/" className="NUnit.VisualStudio.TestAdapter.Tests.DumpXmlTests" name="ThatEmptyContainsHeaders" />
    </UnitTest>

Additional thoughts

I believe it is an issue in the TrxLogger itself: https://github.com/Microsoft/vstest/blob/0341da456744f78bba8ff2a25c1cddfc89c87b08/src/Microsoft.TestPlatform.Extensions.TrxLogger/Utility/Converter.cs#L51 but given that today vstest executable is distributed with a VS only and thus can not be update in reasonable timeframes, would you mind to accept a simple compatibility patch to the adapter to fix the issue?

And vstest still doesn’t support custom traits: https://github.com/Microsoft/vstest/issues/98

Issue Analytics

  • State:open
  • Created 5 years ago
  • Reactions:2
  • Comments:19 (11 by maintainers)

github_iconTop GitHub Comments

2reactions
rprousecommented, Jun 20, 2018

@OsirisTerje did you ever hear back from anyone on this? Should we consider a workaround?

1reaction
klis8commented, Feb 17, 2023
Read more comments on GitHub >

github_iconTop Results From Across the Web

Adapter V3 Release Notes
NUnit3 Test Adapter for Visual Studio - Version 3.17.0 - July 11, 2020 ... 506 Test categories aren't propagated to vstest trx logs....
Read more >
[VsTest - testAssemblies] Publishing test results does not ...
I have reproduced the issue with a single test (to reduce the noise) and the screenshots, logs, .trx files are attached. The logs...
Read more >
Visual Studio 2012 vstest: run tests that do not belong to a ...
I have over 1000 tests, so I am hoping that I don't have to do a search and replace on all the TestMethod...
Read more >
Why is my VSTest Runner test results file not bein...
Specify a logger for test results. For example, to log results into a. Visual Studio Test Results File (TRX) use /logger:trx[;LogFileName=<Defaults to ...
Read more >
dotnet vstest command - .NET CLI
To log results to a Visual Studio Test Results File (TRX), use the trx logger provider. This switch creates a file in the...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found