Error when running on an EKS cluster

Describe the bug I’m running stryker inside an eks container and I get the following error.

Logs [22:55:28 INF] Logging enabled at level Trace Version: 3.5.0

[22:55:29 VRB] CACHE https://api.nuget.org/v3/registration5-gz-semver2/dotnet-stryker/index.json A new version of Stryker.NET (3.7.1) is available. Please consider upgrading using dotnet tool update -g dotnet-stryker

[22:55:29 DBG] Stryker started with options: {“MsBuildPath”: null, “DevMode”: false, “ProjectPath”: “/azp/marketing”, “IsSolutionContext”: true, “WorkingDirectory”: “/azp/marketing”, “OutputPath”: “/azp/marketing/StrykerOutput /2023-04-27.22-55-28”, “ReportPath”: “/azp/marketing/StrykerOutput/2023-04-27.22-55-28/reports”, “ReportFileName”: “mutation-report”, “SolutionPath”: “/azp/marketing/JSM.Marketing.sln”, “TargetFramework”: null, “LogOptions”: { “LogToFile”: true, “LogLevel”: “Verbose”, “$type”: “LogOptions”}, “MutationLevel”: “Basic”, “Thresholds”: {“High”: 80, “Low”: 60, “Break”: 0, “$type”: “Thresholds”}, “AdditionalTimeout”: 5000, “LanguageVersion”: “Default”, “Co ncurrency”: 1, “ProjectUnderTestName”: “JSM.Marketing.Domain”, “TestProjects”: [“/azp/marketing/JSM.Marketing.Test”], “TestCaseFilter”: “”, “Reporters”: [“Progress”, “Html”], “WithBaseline”: false, “BaselineProvider”: “Disk”, “AzureFileStorageUrl”: “”, “AzureFileStorageSas”: “”, “DashboardUrl”: “https://dashboard.stryker-mutator.io”, “DashboardApiKey”: null, “Since”: false, “SinceTarget”: “master”, “DiffIgnoreChanges”: [], “FallbackVersion”: “maste r”, “ModuleName”: “”, “ReportTypeToOpen”: null, “Mutate”: [{“Glob”: {“Tokens”: [{“TrailingPathSeparator”: {“Value”: “/”, “$type”: “PathSeparatorToken”}, “LeadingPathSeparator”: null, “$type”: “WildcardDirectoryToken”}, {"$type ": “WildcardToken”}], “$type”: “Glob”}, “IsExclude”: false, “TextSpans”: [{“Start”: 0, “End”: 2147483647, “Length”: 2147483647, “IsEmpty”: false, “$type”: “TextSpan”}], “$type”: “FilePattern”}], “IgnoredMethods”: [], “Excluded Mutations”: [], “ExcludedLinqExpressions”: [], “OptimizationMode”: “CoverageBasedTest”, “ProjectName”: “”, “ProjectVersion”: “”, “BreakOnInitialTestFailure”: false, “$type”: “StrykerOptions”} [22:55:30 INF] Identifying projects to mutate in /azp/marketing/JSM.Marketing.sln. This can take a while. [22:55:30 DBG] Analysing 5 projects [22:55:30 DBG] Analysing /azp/marketing/JSM.Marketing.Domain/JSM.Marketing.Domain.csproj [22:55:30 DBG] Analysing /azp/marketing/JSM.Marketing.Infrastructure/JSM.Marketing.Infrastructure.csproj [22:55:30 DBG] Analysing /azp/marketing/JSM.Marketing.IoC/JSM.Marketing.IoC.csproj [22:55:30 DBG] Analysing /azp/marketing/JSM.Marketing.Api/JSM.Marketing.Api.csproj [22:55:31 DBG] Analysing /azp/marketing/JSM.Marketing.Test/JSM.Marketing.Test.csproj [22:55:35 INF] Time Elapsed 00:00:06.2003890 Unhandled exception. System.AggregateException: One or more errors occurred. (Could not find build environment) (Could not find build environment) (Could not find build environment) (Could not find build environment) (Could no t find build environment) —> System.InvalidOperationException: Could not find build environment at Buildalyzer.Environment.EnvironmentFactory.GetBuildEnvironment(String targetFramework, EnvironmentOptions options) at Buildalyzer.Environment.EnvironmentFactory.GetBuildEnvironment(String targetFramework) at Buildalyzer.ProjectAnalyzer.Build(String targetFramework) at Buildalyzer.ProjectAnalyzer.Build() at Stryker.Core.Initialisation.ProjectOrchestrator.<>c__DisplayClass7_0.<AnalyzeSolution>b__0(IProjectAnalyzer project) at System.Threading.Tasks.Parallel.<>c__DisplayClass33_02.<ForEachWorker>b__0(Int32 i) at System.Threading.Tasks.Parallel.<>c__DisplayClass19_01.<ForWorker>b__1(RangeWorker& currentWorker, Int32 timeout, Boolean& replicationDelegateYieldedBeforeCompletion) — End of stack trace from previous location — at System.Threading.Tasks.Parallel.<>c__DisplayClass19_01.<ForWorker>b__1(RangeWorker& currentWorker, Int32 timeout, Boolean& replicationDelegateYieldedBeforeCompletion) at System.Threading.Tasks.TaskReplicator.Replica1.ExecuteAction(Boolean& yieldedBeforeCompletion) at System.Threading.Tasks.TaskReplicator.Replica.Execute() — End of inner exception stack trace — at Stryker.Core.Initialisation.ProjectOrchestrator.AnalyzeSolution(StrykerOptions options) at Stryker.Core.Initialisation.ProjectOrchestrator.MutateProjects(StrykerOptions options, IReporter reporters)+MoveNext() at System.Collections.Generic.List1..ctor(IEnumerable1 collection) at System.Linq.Enumerable.ToList[TSource](IEnumerable1 source) at Stryker.Core.StrykerRunner.RunMutationTest(IStrykerInputs inputs, ILoggerFactory loggerFactory, IProjectOrchestrator projectOrchestrator) at Stryker.CLI.StrykerCli.RunStryker(IStrykerInputs inputs) in /_/src/Stryker.CLI/Stryker.CLI/StrykerCLI.cs:line 93 at Stryker.CLI.StrykerCli.<>c__DisplayClass10_0.<Run>b__0() in /_/src/Stryker.CLI/Stryker.CLI/StrykerCLI.cs:line 68 at McMaster.Extensions.CommandLineUtils.CommandLineApplication.<>c__DisplayClass143_0.<OnExecute>b__0(CancellationToken _) at McMaster.Extensions.CommandLineUtils.CommandLineApplication.ExecuteAsync(String[] args, CancellationToken cancellationToken) at McMaster.Extensions.CommandLineUtils.CommandLineApplication.Execute(String[] args) at Stryker.CLI.StrykerCli.Run(String[] args) in /_/src/Stryker.CLI/Stryker.CLI/StrykerCLI.cs:line 74 at Stryker.CLI.Program.Main(String[] args) in /_/src/Stryker.CLI/Stryker.CLI/Program.cs:line 14 ---> (Inner Exception #1) System.InvalidOperationException: Could not find build environment at Buildalyzer.Environment.EnvironmentFactory.GetBuildEnvironment(String targetFramework, EnvironmentOptions options) at Buildalyzer.Environment.EnvironmentFactory.GetBuildEnvironment(String targetFramework) at Buildalyzer.ProjectAnalyzer.Build(String targetFramework) at Buildalyzer.ProjectAnalyzer.Build() at Stryker.Core.Initialisation.ProjectOrchestrator.<>c__DisplayClass7_0.<AnalyzeSolution>b__0(IProjectAnalyzer project) at System.Threading.Tasks.Parallel.<>c__DisplayClass33_02.<ForEachWorker>b__0(Int32 i) at System.Threading.Tasks.Parallel.<>c__DisplayClass19_01.<ForWorker>b__1(RangeWorker& currentWorker, Int32 timeout, Boolean& replicationDelegateYieldedBeforeCompletion) --- End of stack trace from previous location --- at System.Threading.Tasks.Parallel.<>c__DisplayClass19_01.<ForWorker>b__1(RangeWorker& currentWorker, Int32 timeout, Boolean& replicationDelegateYieldedBeforeCompletion) at System.Threading.Tasks.TaskReplicator.Replica`1.ExecuteAction(Boolean& yieldedBeforeCompletion) at System.Threading.Tasks.TaskReplicator.Replica.Execute()<—

—> (Inner Exception #2) System.InvalidOperationException: Could not find build environment at Buildalyzer.Environment.EnvironmentFactory.GetBuildEnvironment(String targetFramework, EnvironmentOptions options) at Buildalyzer.Environment.EnvironmentFactory.GetBuildEnvironment(String targetFramework) at Buildalyzer.ProjectAnalyzer.Build(String targetFramework) at Buildalyzer.ProjectAnalyzer.Build() at Stryker.Core.Initialisation.ProjectOrchestrator.<>c__DisplayClass7_0.<AnalyzeSolution>b__0(IProjectAnalyzer project) at System.Threading.Tasks.Parallel.<>c__DisplayClass33_02.<ForEachWorker>b__0(Int32 i) at System.Threading.Tasks.Parallel.<>c__DisplayClass19_01.<ForWorker>b__1(RangeWorker& currentWorker, Int32 timeout, Boolean& replicationDelegateYieldedBeforeCompletion) — End of stack trace from previous location — at System.Threading.Tasks.Parallel.<>c__DisplayClass19_01.<ForWorker>b__1(RangeWorker& currentWorker, Int32 timeout, Boolean& replicationDelegateYieldedBeforeCompletion) at System.Threading.Tasks.TaskReplicator.Replica1.ExecuteAction(Boolean& yieldedBeforeCompletion) at System.Threading.Tasks.TaskReplicator.Replica.Execute()<—

—> (Inner Exception #3) System.InvalidOperationException: Could not find build environment at Buildalyzer.Environment.EnvironmentFactory.GetBuildEnvironment(String targetFramework, EnvironmentOptions options) at Buildalyzer.Environment.EnvironmentFactory.GetBuildEnvironment(String targetFramework) at Buildalyzer.ProjectAnalyzer.Build(String targetFramework) at Buildalyzer.ProjectAnalyzer.Build() at Stryker.Core.Initialisation.ProjectOrchestrator.<>c__DisplayClass7_0.<AnalyzeSolution>b__0(IProjectAnalyzer project) at System.Threading.Tasks.Parallel.<>c__DisplayClass33_02.<ForEachWorker>b__0(Int32 i) at System.Threading.Tasks.Parallel.<>c__DisplayClass19_01.<ForWorker>b__1(RangeWorker& currentWorker, Int32 timeout, Boolean& replicationDelegateYieldedBeforeCompletion) — End of stack trace from previous location — at System.Threading.Tasks.Parallel.<>c__DisplayClass19_01.<ForWorker>b__1(RangeWorker& currentWorker, Int32 timeout, Boolean& replicationDelegateYieldedBeforeCompletion) at System.Threading.Tasks.TaskReplicator.Replica1.ExecuteAction(Boolean& yieldedBeforeCompletion) at System.Threading.Tasks.TaskReplicator.Replica.Execute()<—

—> (Inner Exception #4) System.InvalidOperationException: Could not find build environment at Buildalyzer.Environment.EnvironmentFactory.GetBuildEnvironment(String targetFramework, EnvironmentOptions options) at Buildalyzer.Environment.EnvironmentFactory.GetBuildEnvironment(String targetFramework) at Buildalyzer.ProjectAnalyzer.Build(String targetFramework) at Buildalyzer.ProjectAnalyzer.Build() at Stryker.Core.Initialisation.ProjectOrchestrator.<>c__DisplayClass7_0.<AnalyzeSolution>b__0(IProjectAnalyzer project) at System.Threading.Tasks.Parallel.<>c__DisplayClass33_02.<ForEachWorker>b__0(Int32 i) at System.Threading.Tasks.Parallel.<>c__DisplayClass19_01.<ForWorker>b__1(RangeWorker& currentWorker, Int32 timeout, Boolean& replicationDelegateYieldedBeforeCompletion) — End of stack trace from previous location — at System.Threading.Tasks.Parallel.<>c__DisplayClass19_01.<ForWorker>b__1(RangeWorker& currentWorker, Int32 timeout, Boolean& replicationDelegateYieldedBeforeCompletion) at System.Threading.Tasks.TaskReplicator.Replica1.ExecuteAction(Boolean& yieldedBeforeCompletion) at System.Threading.Tasks.TaskReplicator.Replica.Execute()<—

Expected behavior A clear and concise description of what you expected to happen.

Desktop (please complete the following information):

• OS: ubuntu(EKS Cluster)
• Type of project: Web API
• Framework Version: Core 6.0
• Stryker Version: 3.5.0

Additional context Add any other context about the problem here.

[phmonte]

@rouke-broersma I created the issue in the Buildalyzer repository, in my case the cache didn’t work because the Process doesn’t call the .bashrc file where I created the cache, so I generated a new build of Stryker using an altered version of Buildalyzer and now it works. Let’s wait for the Buildalyzer team, if they approve the change I can do the PR on Buildalyzer and Stryker with the correction.

PS: It took me hours to understand why my version didn’t work, your post and evaluating the pipeline saved me (about vstest files).

https://github.com/microsoft/vstest/issues/1948

[phmonte]

I will try to reproduce the problem following your tips and I will post the updates here.

Thank you very much