No more Benchmark runs will be launched as NO measurements were obtained from the previous run!
See original GitHub issueUsing the exact dependencies (NUnit 3.7.1, Benchmark.NET 0.10.8 and CodeJam.PerfTests.NUnit 0.1.4-beta) and the wiki example:
[Category("PerfTests: NUnit examples")]
[CompetitionAnnotateSources] // Opt-in feature: source annotations.
public class SimplePerfTest
{
private const int Count = 200;
// Perf test runner method.
[Test]
public void RunSimplePerfTest() => Competition.Run(this);
// Baseline competition member.
// All relative metrics will be compared with metrics of the baseline method.
[CompetitionBaseline]
public void Baseline() => Thread.SpinWait(Count);
// Competition member #1. Should take ~3x more time to run.
[CompetitionBenchmark]
public void SlowerX3() => Thread.SpinWait(3 * Count);
// Competition member #2. Should take ~5x more time to run.
[CompetitionBenchmark]
public void SlowerX5() => Thread.SpinWait(5 * Count);
// Competition member #3. Should take ~7x more time to run.
[CompetitionBenchmark]
public void SlowerX7() => Thread.SpinWait(7 * Count);
}
I get the following:
Test completed with errors, details below.
Errors:
* Run #1: No result reports for benchmarks: Baseline, SlowerX3, SlowerX5, SlowerX7.
at CodeJam.PerfTests.Running.Core.NUnitCompetitionRunner.ReportExecutionErrors(String messages, CompetitionState competitionState) in C:\Users\igors\Source\Repos\CodeJam\PerfTests\src-NUnit\Running.Core\NUnitCompetitionRunner.cs:line 81
at CodeJam.PerfTests.Running.Core.CompetitionRunnerBase.RunCore(Type benchmarkType, ICompetitionConfig competitionConfig, CompetitionFeatures competitionFeatures) in C:\Users\igors\Source\Repos\CodeJam\PerfTests\src\[L5_AllTogether]\Running.Core\CompetitionRunnerBase.cs:line 233
BenchmarkDotNet=v0.10.8, OS=Windows 10 Redstone 1 (10.0.14393)
Processor=Intel Core i7-7700 CPU 3.60GHz (Kaby Lake), ProcessorCount=8
Frequency=3515621 Hz, Resolution=284.4448 ns, Timer=TSC
[Host] : Clr 4.0.30319.42000, 64bit RyuJIT-v4.7.2558.0
Job=Competition Force=False Toolchain=InProcessToolchain
InvocationCount=256 LaunchCount=1 RunStrategy=Throughput
TargetCount=256 UnrollFactor=16 WarmupCount=128
AdjustMetrics=True
Method | Mean | StdDev | Scaled | Scaled-StdDev | GcAllocations |
--------- |-----:|-------:|-------:|--------------:|--------------:|
Baseline | NA | NA | 1.00 | 0.00 | ? |
SlowerX3 | NA | NA | ? | ? | ? |
SlowerX5 | NA | NA | ? | ? | ? |
SlowerX7 | NA | NA | ? | ? | ? |
Benchmarks with issues:
SimplePerfTest.Baseline: Competition(Force=False, Toolchain=InProcessToolchain, InvocationCount=256, LaunchCount=1, RunStrategy=Throughput, TargetCount=256, UnrollFactor=16, WarmupCount=128)
SimplePerfTest.SlowerX3: Competition(Force=False, Toolchain=InProcessToolchain, InvocationCount=256, LaunchCount=1, RunStrategy=Throughput, TargetCount=256, UnrollFactor=16, WarmupCount=128)
SimplePerfTest.SlowerX5: Competition(Force=False, Toolchain=InProcessToolchain, InvocationCount=256, LaunchCount=1, RunStrategy=Throughput, TargetCount=256, UnrollFactor=16, WarmupCount=128)
SimplePerfTest.SlowerX7: Competition(Force=False, Toolchain=InProcessToolchain, InvocationCount=256, LaunchCount=1, RunStrategy=Throughput, TargetCount=256, UnrollFactor=16, WarmupCount=128)
============= SimplePerfTest =============
// ? Squid.Strategies.Tests.Collect2.SimplePerfTest, Squid.Strategies.Tests
---- Run 1, total runs (expected): 1 -----
ExitCode != 0
No more Benchmark runs will be launched as NO measurements were obtained from the previous run!
// Exception: System.InvalidOperationException: Sequence contains no elements
at System.Linq.Enumerable.Last[TSource](IEnumerable`1 source)
at BenchmarkDotNet.Running.BenchmarkRunnerCore.Execute(ILogger logger, Benchmark benchmark, IToolchain toolchain, BuildResult buildResult, IConfig config, IResolver resolver, GcStats& gcStats)
at BenchmarkDotNet.Running.BenchmarkRunnerCore.Run(Benchmark benchmark, ILogger logger, IConfig config, String rootArtifactsFolderPath, Func`2 toolchainProvider, IResolver resolver)
ExitCode != 0
No more Benchmark runs will be launched as NO measurements were obtained from the previous run!
// Exception: System.InvalidOperationException: Sequence contains no elements
at System.Linq.Enumerable.Last[TSource](IEnumerable`1 source)
at BenchmarkDotNet.Running.BenchmarkRunnerCore.Execute(ILogger logger, Benchmark benchmark, IToolchain toolchain, BuildResult buildResult, IConfig config, IResolver resolver, GcStats& gcStats)
at BenchmarkDotNet.Running.BenchmarkRunnerCore.Run(Benchmark benchmark, ILogger logger, IConfig config, String rootArtifactsFolderPath, Func`2 toolchainProvider, IResolver resolver)
ExitCode != 0
No more Benchmark runs will be launched as NO measurements were obtained from the previous run!
// Exception: System.InvalidOperationException: Sequence contains no elements
at System.Linq.Enumerable.Last[TSource](IEnumerable`1 source)
at BenchmarkDotNet.Running.BenchmarkRunnerCore.Execute(ILogger logger, Benchmark benchmark, IToolchain toolchain, BuildResult buildResult, IConfig config, IResolver resolver, GcStats& gcStats)
at BenchmarkDotNet.Running.BenchmarkRunnerCore.Run(Benchmark benchmark, ILogger logger, IConfig config, String rootArtifactsFolderPath, Func`2 toolchainProvider, IResolver resolver)
ExitCode != 0
No more Benchmark runs will be launched as NO measurements were obtained from the previous run!
// Exception: System.InvalidOperationException: Sequence contains no elements
at System.Linq.Enumerable.Last[TSource](IEnumerable`1 source)
at BenchmarkDotNet.Running.BenchmarkRunnerCore.Execute(ILogger logger, Benchmark benchmark, IToolchain toolchain, BuildResult buildResult, IConfig config, IResolver resolver, GcStats& gcStats)
at BenchmarkDotNet.Running.BenchmarkRunnerCore.Run(Benchmark benchmark, ILogger logger, IConfig config, String rootArtifactsFolderPath, Func`2 toolchainProvider, IResolver resolver)
There are no any results runs
There are no any results runs
There are no any results runs
There are no any results runs
Benchmarks with issues:
SimplePerfTest.Baseline: Competition(Force=False, Toolchain=InProcessToolchain, InvocationCount=256, LaunchCount=1, RunStrategy=Throughput, TargetCount=256, UnrollFactor=16, WarmupCount=128)
SimplePerfTest.SlowerX3: Competition(Force=False, Toolchain=InProcessToolchain, InvocationCount=256, LaunchCount=1, RunStrategy=Throughput, TargetCount=256, UnrollFactor=16, WarmupCount=128)
SimplePerfTest.SlowerX5: Competition(Force=False, Toolchain=InProcessToolchain, InvocationCount=256, LaunchCount=1, RunStrategy=Throughput, TargetCount=256, UnrollFactor=16, WarmupCount=128)
SimplePerfTest.SlowerX7: Competition(Force=False, Toolchain=InProcessToolchain, InvocationCount=256, LaunchCount=1, RunStrategy=Throughput, TargetCount=256, UnrollFactor=16, WarmupCount=128)
// ! #1.1 01.442s, ExecutionError@Analyser: No result reports for benchmarks: Baseline, SlowerX3, SlowerX5, SlowerX7.
// ! Hint: Ensure that benchmarks were run successfully and did not throw any exceptions.
// ! Breaking competition execution. High severity error occured.
Issue Analytics
- State:
- Created 6 years ago
- Comments:14 (14 by maintainers)
Top Results From Across the Web
No more Benchmark runs will be launched as ...
Hi there, I've been attempting to run some test which require retrieving objects via indexers using unsafe code and I keep getting a...
Read more >SPECpower_ssj2008 Run and Reporting Rules
The intent in defining and automating active idle power measurement within the SPECpower_ssj2008 benchmark is to prevent manipulation of idle ...
Read more >Run benchmarks in Continuous Integration | App quality
You can run benchmarks in Continuous Integration (CI) to track performance over time, and recognize performance regressions (or improvements) before your ...
Read more >Benchmarking in Golang: Improving function performance
A benchmark is a type of function that executes a code segment multiple times and compares each output against a standard, assessing the ......
Read more >Benchmark | Jetpack
Accurately measure your code's performance within Android Studio. Latest Update, Stable Release, Release Candidate, Beta Release, Alpha Release. August 9, 2023 ...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
@ig-sinicyn Check whether you can require lower version for any dependency like NUnit 3.0.0 instead of 3.12 and so on.
Let’s close it. Perftest are currently dead as I have no time to revive them:)