Process is terminated due to StackOverflowException. (VsTest error)

Hi,

When I run stryker-net with my application, I received the error below.

Environment: Windows 10 .Net 4.7.1 (WPF app) Stryker-net 0.15.0

Logs: https://we.tl/t-629yoPTUqL

Sorry I needed to share logs through WeTransfer since logs are greater than 10MB.

Regards,

Gaetan

Exception:

2020-01-07T00:29:00.2060176-05:00  [VRB] Runner 3: [Debug] Logging TestHost Diagnostics in file: C:\Sources\Dev\CAD\CAD.Wpf.Logic.Test\StrykerOutput\2020-01-06.13-26-33\logs\vstest-log.host.20-01-07_00-28-59_92821_6.txt (346d8624)
2020-01-07T00:29:00.3854577-05:00  [VRB] Runner 3: [Debug] NUnit Adapter 3.16.0.0: Test execution started (7b149e97)
2020-01-07T00:29:00.4283105-05:00  [VRB] Runner 3: [Debug] Running selected tests in C:\Sources\Dev\CAD\CAD.Wpf.Logic.Test\bin\Debug\Emergensys.CAD.Wpf.Logic.Test.dll (a5b5683d)
2020-01-07T00:29:01.5272231-05:00  [VRB] Runner 3: [Debug]    NUnit3TestExecutor converted 7 of 7 NUnit test cases (5011a05f)
2020-01-07T00:29:06.0424410-05:00  [DBG] Runner 3:  (VsTest error) (671cf401)
2020-01-07T00:29:06.0425337-05:00  [DBG] Runner 3: Process is terminated due to StackOverflowException. (VsTest error) (d263dc55)
2020-01-07T00:29:10.3523292-05:00  [VRB] Runner 3: [Warning] The active test run was aborted. Reason: Test host process crashed (1bd06da0)
2020-01-07T00:29:10.3756252-05:00  [WRN] VsTest error occured. Please report the error at https://github.com/stryker-mutator/stryker-net/issues (d88e9c55)
System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host
   at System.Net.Sockets.Socket.Receive(Byte[] buffer, Int32 offset, Int32 size, SocketFlags socketFlags)
   at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
   --- End of inner exception stack trace ---
   at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
   at System.IO.Stream.ReadByte()
   at System.IO.BinaryReader.ReadByte()
   at System.IO.BinaryReader.Read7BitEncodedInt()
   at System.IO.BinaryReader.ReadString()
   at Microsoft.VisualStudio.TestPlatform.CommunicationUtilities.LengthPrefixCommunicationChannel.NotifyDataAvailable()
   at Microsoft.VisualStudio.TestPlatform.CommunicationUtilities.TcpClientExtensions.MessageLoopAsync(TcpClient client, ICommunicationChannel channel, Action`1 errorHandler, CancellationToken cancellationToken)
2020-01-07T00:29:10.3758441-05:00  [VRB] Testrun aborted due to timeout (f49baa08)
2020-01-07T00:29:10.3807541-05:00  [DBG] Testing Negate expression on line 32: '!_applicationContext.IsEditingCallcard' ==> '!(!_applicationContext.IsEditingCallcard)'. (1c526302)
2020-01-07T00:29:10.4183968-05:00  [DBG] Runner 3: Testing 6257: Negate expression against: Emergensys.CAD.Wpf.Logic.Test.ViewModel.CallCard.Command.CallCardViewModelCommandCreationModeTest.ExecutingDeferShouldNotThrowIfParameterIsNull, Emergensys.CAD.Wpf.Logic.Test.ViewModel.CallCard.Command.OpenDeferWindowCommandTests.GivenAnotherCallCardInEdition_ExecuteShouldAskToSaveOrCancelCurrentEdition, Emergensys.CAD.Wpf.Logic.Test.ViewModel.CallCard.Command.OpenDeferWindowCommandTests.GivenCallCardDirty_ExecuteShouldSaveFirst, Emergensys.CAD.Wpf.Logic.Test.ViewModel.CallCard.Command.OpenDeferWindowCommandTests.GivenCallCardVmNotOpen_ExecuteShouldOpenItFirst, Emergensys.CAD.Wpf.Logic.Test.ViewModel.CallCard.Command.OpenDeferWindowCommandTests.OnReservationSucceedShoudlFocusOpenedDeferWindow, Emergensys.CAD.Wpf.Logic.Test.ViewModel.CallCard.Command.OpenDeferWindowCommandTests.OnReservationSucceedShoudlOpenDeferWindow, Emergensys.CAD.Wpf.Logic.Test.ViewModel.CallCard.Command.OpenDeferWindowCommandTests.ShouldReserverAllWhenOpeningDefer. (5fab4b08)
2020-01-07T00:29:10.4184711-05:00  [DBG] Runner 3: VsTest runsettings set to: <RunSettings>

[rouke-broersma]

@ggingras I’m gonna go ahead and close this for now. If the issue persists even if you refresh the console window feel free to comment and I’ll reopen the issue.

[dupdob]

Actually there was a bunch of stack overflows during the night, so I don’t think this is the actual issue. There is no actual error in the log. Everything ran fine until 9:27 AM. I need to ask this: are you sure you did not inadvertently click on the console window? Any action that is interpreted as a selection attempt by Windows locks the console and consequently blocks the currently running application (i.e. Stryker here). If this happens, you just need to press Esc to restore operations.

Could you please try again and see what happens?

I went through the logs. Here are a few interesting numbers:

• The initial test run is around 4 minutes (3’ 50", this is quite long)
• ~9K tests are detected (8928)
• ~24K mutants are generated (24032)

Without any optimisation this implies an estimated total running time of … ONE MONTH!. Actually 768 hours or 32 days. Actual time may be twice that or even more. It could also be only half or a third of this. It depends on how many mutants are killed and how fast. As 4 runners are used (as 4 cores are available), we can divide this by 4. So we have an initial runtime of roughly 8 days. Coverage based optimisation eliminated 97.74% of tests. ⇒ This leads to an ideal run time of around 3 hours.

But it appears your actual runtime is significantly higher. This is normal because my back of hand calculations assume that tests have a similar run time. This is rarely the case. For example end to end or integration tests take far more time than traditional unit tests. Judging by what I saw in the logs, it looks like you have several ent to end (or end to end like) tests. And end to end tests cover many mutants => so end to end tests are run against a log of mutants.

As a conclusion

I advise you/your team to split this project in a couple (or more) assemblies and test them separately, this would significantly reduce run time.

Thanks for your logs, they will help us improve the experience on large projects.