[iOS] Infrastructure is failing to report successful test run

Build

https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=89051

Build leg reported

System.Runtime.Tests.WorkItemExecution

Pull Request

https://github.com/dotnet/runtime/pull/78288

Action required for the engineering services team

To triage this issue (First Responder / @dotnet/dnceng):

• Open the failing build above and investigate
• Add a comment explaining your findings

If this is an issue that is causing build breaks across multiple builds and would get benefit from being listed on the build analysis check, follow the next steps:

1. Add the label “Known Build Error”
2. Edit this issue and add an error string in the Json below that can help us match this issue with future build breaks. You should use the known issues documentation

{
   "ErrorMessage" : "[TCP tunnel] Xamarin.Hosting: Failed to connect to port",
   "BuildRetry": true
}

Additional information about the issue reported

System.Runtime.Tests are reported as failing. It looks like a failure in the reporting infrastructure. System.Runtime.Tests succeeded according to the net.dot.System.Runtime.Tests.log file:

Tests run: 50259 Passed: 50025 Inconclusive: 0 Failed: 0 Ignored: 133 Skipped: 101
Killing process 95249 as it was cancelled
[TerminateWithSuccess]

Report

Build	Definition	Test	Pull Request
89138	dotnet/runtime	System.Runtime.Tests.WorkItemExecution	dotnet/runtime#78593

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
1	1	1

[premun]

Seems like the new fixes are already working! There are the TCP problems being detected:

But what is more important is that the retries happening for these cases now dropped the overall failure rate of AppleTV jobs to 0:

cc @AlitzelMendez @ilyas1974

[steveisok]

Thank you. Do you think that the following is something worthy of a try?

We’ve tried variations of this with the same result, so no tbh. I think, as you pointed out, it’s an issue w/ mlaunch/xcode/osx or something within runtime. I don’t believe it’s the latter.