question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

MSTest 2.2.4 does not work with [DataTestMethod] [DynamicData] and a complex object

See original GitHub issue

Description

Everything was working fine in 2.2.3. Then we updated to 2.2.4 and it does not work anymore. So we have some classes with [DataTestMethod] and some dynamic data that we feed into it. The tests were green in 2.2.3 and upgrading to 2.2.4 break them with this error:

C:\Users\mme\Documents\GitHub\MsTest224DataTestBug\MsTest224DataTestBug>dotnet test MsTest224DataTestBug.csproj
 Determining projects to restore...
 Restored C:\Users\mme\Documents\GitHub\MsTest224DataTestBug\MsTest224DataTestBug\MsTest224DataTestBug.csproj (in 529 ms).
 MsTest224DataTestBug -> C:\Users\mme\Documents\GitHub\MsTest224DataTestBug\MsTest224DataTestBug\bin\Debug\net5.0\MsTest224DataTestBug.dll
Test run for C:\Users\mme\Documents\GitHub\MsTest224DataTestBug\MsTest224DataTestBug\bin\Debug\net5.0\MsTest224DataTestBug.dll (.NETCoreApp,Version=v5.0)
Microsoft (R) Test Execution Command Line Tool Version 16.10.0
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
An exception occurred while invoking executor 'executor://mstestadapter/v2': Type 'MsTest224DataTestBug.MyStringContainer' with data contract name 'MyStringContainer:http://schemas.datacontract.org/2004/07/MsTest224DataTestBug' is not expected. Add any types not known statically to the list of known types - for example, by using the KnownTypeAttribute attribute or by adding them to the list of known types passed to DataContractSerializer.
Stack trace:
  at System.Runtime.Serialization.XmlObjectSerializerWriteContext.SerializeAndVerifyType(DataContract dataContract, XmlWriterDelegator xmlWriter, Object obj, Boolean verifyKnownType, RuntimeTypeHandle declaredTypeHandle, Type declaredType)
  at System.Runtime.Serialization.Json.XmlObjectSerializerWriteContextComplexJson.SerializeWithXsiTypeAtTopLevel(DataContract dataContract, XmlWriterDelegator xmlWriter, Object obj, RuntimeTypeHandle originalDeclaredTypeHandle, Type graphType)
  at System.Runtime.Serialization.Json.DataContractJsonSerializerImpl.InternalWriteObjectContent(XmlWriterDelegator writer, Object graph)
  at System.Runtime.Serialization.Json.DataContractJsonSerializerImpl.InternalWriteObject(XmlWriterDelegator writer, Object graph)
  at System.Runtime.Serialization.XmlObjectSerializer.InternalWriteObject(XmlWriterDelegator writer, Object graph, DataContractResolver dataContractResolver)
  at System.Runtime.Serialization.XmlObjectSerializer.WriteObjectHandleExceptions(XmlWriterDelegator writer, Object graph, DataContractResolver dataContractResolver)
  at System.Runtime.Serialization.Json.DataContractJsonSerializerImpl.WriteObject(XmlDictionaryWriter writer, Object graph)
  at System.Runtime.Serialization.Json.DataContractJsonSerializerImpl.WriteObject(Stream stream, Object graph)
  at Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.Helpers.DataSerializationHelper.Serialize(Object[] data)
  at Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.ObjectModel.UnitTestElement.ToTestCase()
  at Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.UnitTestDiscoverer.SendTestCases(String source, IEnumerable`1 testElements, ITestCaseDiscoverySink discoverySink, IDiscoveryContext discoveryContext, IMessageLogger logger)
  at Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.UnitTestDiscoverer.DiscoverTestsInSource(String source, IMessageLogger logger, ITestCaseDiscoverySink discoverySink, IDiscoveryContext discoveryContext)
  at Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.Execution.TestExecutionManager.RunTests(IEnumerable`1 sources, IRunContext runContext, IFrameworkHandle frameworkHandle, TestRunCancellationToken cancellationToken)
  at Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.MSTestExecutor.RunTests(IEnumerable`1 sources, IRunContext runContext, IFrameworkHandle frameworkHandle)
  at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Execution.RunTestsWithSources.InvokeExecutor(LazyExtension`2 executor, Tuple`2 executorUriExtensionTuple, RunContext runContext, IFrameworkHandle frameworkHandle)
  at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Execution.BaseRunTests.RunTestInternalWithExecutors(IEnumerable`1 executorUriExtensionMap, Int64 totalTests)


Additionally, path to test adapters can be specified using /TestAdapterPath command. Example  /TestAdapterPath:<pathToCustomAdapters>.

Steps to reproduce

Here is a full sample reproducing the bug: https://github.com/TheFireCookie/MsTest224DataTestBug In this repo, there is a folder MsTest223DataTestBug in which the command ‘dotnet test’ works correctly, the other folder is MsTest224DataTestBug and in this one, it’s exactly the same code except I’ve upgraded MSTest to 2.2.4 and there a ‘dotnet test’ fails with the error written earlier.

Expected behavior

No regression and we can still use [DynamicData] with complex objects

Actual behavior

We cannot use anymore [DynamicData] with complex objects, only simple types like string, int etc

Environment

Tested and reproduced on Windows 10 and CentOS 7.9. It’s running on .NET 5.0.

AB#1334620

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Reactions:28
  • Comments:19 (9 by maintainers)

github_iconTop GitHub Comments

12reactions
Haploiscommented, May 27, 2021

Thank you for the report. This bug is caused because of a change in how we handle attributes derived from ITestDataSource.

I am working on a fix, and will release a preview Monday for this bug.

5reactions
japjcommented, Jun 16, 2021

@Haplois

With #864, last failing case is fixed too. We also added some tests.

Is there a timeline on when 2.2.5 is planned to be released?

Read more comments on GitHub >

github_iconTop Results From Across the Web

Test data "disappearing" when using MSTest DynamicData
I have a MS Test project for the library. I use dynamic data heavily and recently noticed that not all of my test...
Read more >
StackOverflowException running tests with DynamicData ...
A StackOverflowException occurs running a DataTestMethod test using DynamicData when the target POCO class includes a JToken as a Dictionary ...
Read more >
Improving complex MSTest unit tests with DynamicData
With MSTest's DynamicData attribute we can specify a helper property or method to retrieve test data from. C# treats both of those sources...
Read more >
Create Data-Driven Unit Tests - Visual Studio (Windows)
You can use the Microsoft unit test framework (MSTest) for managed code to set up a unit test method to retrieve values from...
Read more >
MSTest v2: Data tests
If your data cannot be set into an attribute parameter (non-constant values or complex objects), you can use the [DynamicData] attribute.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found