question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Parallel TestContext output is not set to the correct test

See original GitHub issue

Description

When outputting text via TestContext and running the tests in parallel, the output is not set to the correct test

Steps to reproduce

Have multiple classes which output text and check the resulting trx file. Note: Running the tests via Visual Studio does not seem to produce this, but when running the dotnet command manually dotnet test MSTest_TestContextIssues.dll -l:trx;LogFileName=TestResults.xml

Expected behavior

The output is set to the correct test

Actual behavior

My test is simply outputting the class name and the test method and in a single test it has outputted:

        <StdOut>Std: UnitTest4 - TestMethod1&#xD;
Std: UnitTest7 - TestMethod1&#xD;
Std: UnitTest5 - TestMethod1&#xD;
Std: UnitTest8 - TestMethod1&#xD;
Std: UnitTest6 - TestMethod1&#xD;
Std: UnitTest2 - TestMethod1&#xD;
Std: UnitTest3 - TestMethod1&#xD;
Std: UnitTest1 - TestMethod1&#xD;
&#xD;


TestContext Messages:
TC: UnitTest1 - \u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000TestMethod1\u0000\u0000&#xD;
: UnitTest7 - TestMethod1&#xD;
TC: UnitTest5 - TestMethod1&#xD;
TC: UnitTest6 - TestMethod1&#xD;
TC: UnitTest8 - TestMethod1&#xD;
TC: UnitTest2 - TestMethod1&#xD;
TC: UnitTest3 - TestMethod1</StdOut>

Environment

The fix provided for v2.2.9 seems to have fixed it for std output when running the tests in Visual Studio.

But the issue still persists when running with dotnet test

Refer sample project here

Issue Analytics

  • State:closed
  • Created a year ago
  • Reactions:1
  • Comments:7 (6 by maintainers)

github_iconTop GitHub Comments

3reactions
Evangelinkcommented, May 22, 2023

Hi there,

I just looked at the code to understand what’s going-on and how things are setup and I understand what’s the issue. While the fix is not technically super challenging it risks to impact lot of places in the codebase. I will try to confirm if I can avoid touching any public API in this case we could work on it for some 3.xx release otherwise we will have to postpone it for a 4.xx release.

Technical note: We need to replace the various stream globally and only once with a custom class. Before the call of any test, we would use AsyncLocal feature to set a unique identifier (test ID?) that would be used in the custom class to classify received (for example adding to a ConcurrentDictionary). At the end of the test execution, we can retrieve the entry and add it to the test result.

1reaction
Evangelinkcommented, Nov 9, 2022

@StevenDenman @krijohan I have managed to reproduce the issue. Thank you!

Read more comments on GitHub >

github_iconTop Results From Across the Web

Setting output file for tests with SetUp method
Hello i have a class full of testcase -s.I want to pre-set each testcase output file depending on his identifier or name. class...
Read more >
MSTest V2: in-assembly parallel test execution
I ask because when I switched to MSTest V2, our unit tests executed in parallel fashion flawlessly, but our integration tests could not...
Read more >
Output from NUnit test runner is broken when running tests ...
The output from NUnit tests is completely broken when running tests in parallel. Some tests doesn't show any output, some output is shown...
Read more >
TestContext
The TestContext class allows tests to access certain information about the execution context. Note. It's important to remember that "test" in NUnit may...
Read more >
Test runner | Node.js v20.5.1 Documentation
If any tests fail, the process exit code is set to 1 . Subtests#. The test context's test() method allows subtests to be...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found