Some(0) == Some(1) too slow on first execution

High-level Summary

The first time an application executes Some(0) == Some(1), the time required is extremely long. By “extremely long”, I mean the time required to first execute 0 == 1 is a few orders of magnitude smaller than the time required to first execute Some(0) == Some(1). I provide exact execution times and ratios below.

I narrowed the problem down to this line of EqOptional<,,,>.Equals, which just calls EqDefault<>.Equals. I think the optimization of that method needs to be improved.

Testing

Normally in performance testing, the proper testing methodology is to record the time required to execute the code in question n times and then divide by n to get the average runtime. I want to stress that this is not the problem. Only the first execution is taking a long time, so computing an average of many executions will make it seem like there is no large slowdown. When only considering a single execution, the exact execution time is not so trustworthy, but we have to make due. I will focus on the relative execution time between two operations and round generously.

Consider this test, which compares the execution time of 0 == 1 with that of EqDefault<int>.Equals(0, 1).

private readonly ITestOutputHelper output;
public Appendable(ITestOutputHelper output) => this.output = output;

[Fact]
public void PerformanceTest()
{
    _LogPerformanceTests();
    output.WriteLine("AGAIN");
    _LogPerformanceTests();
}

private void _LogPerformanceTests()
{
    var intInt = _PerformanceTest("int == int", () => 0 == 1);
    var eqDefaultIntEquals = _PerformanceTest("EqDefault<int>.Equals", () => TypeClass.equals<EqDefault<int>, int>(0, 1));
    output.WriteLine($"{eqDefaultIntEquals / intInt} factor slowdown for EqDefault<int>.Equals");
}

private double _PerformanceTest<A>(string name, Func<A> f)
{
    var n = 1;
    var stopwatch = Stopwatch.StartNew();
    Enumerable.Range(1, n).Iter(_ => f());
    var averageMilliseconds = stopwatch.Elapsed.TotalMilliseconds / n;
    output.WriteLine($"{averageMilliseconds} for {name}");
    return averageMilliseconds;
}

Here is the output when executed in a few different scenarios.

As a test in the Language Ext solution with the most recent commit on master (which among released versions most closely corresponds to version 3.1.14):

1.3205 for int == int 191.783 for EqDefault<int>.Equals 145.235138205225 factor slowdown for EqDefault<int>.Equals AGAIN 0.0038 for int == int 0.0055 for EqDefault<int>.Equals 1.44736842105263 factor slowdown for EqDefault<int>.Equals

As a test in the Language Ext solution with version 3.0.0:

2.4726 for int == int 225.4662 for EqDefault<int>.Equals 91.1858772142684 factor slowdown for EqDefault<int>.Equals AGAIN 0.0029 for int == int 0.0064 for EqDefault<int>.Equals 2.20689655172414 factor slowdown for EqDefault<int>.Equals

In an (otherwise empty) application targeting .Net 4.6.1 with a NuGet package of Language Ext 3.0.0:

28.3751 for int == int 1813.1669 for EqDefault<int>.Equals 63.8999298680886 factor slowdown for EqDefault<int>.Equals AGAIN 0.0017 for int == int 0.0064 for EqDefault<int>.Equals 3.76470588235294 factor slowdown for EqDefault<int>.Equals

In my application for work targeting .Net 4.6.1 with NuGet package of Language Ext 3.0.0 running through Mono in an embedded Linux distribution on relatively weak hardware:

79.64710000000001 for int == int 106831.62550000001 for EqDefault<int>.Equals 1341.3121821133475 factor slowdown for EqDefault<int>.Equals AGAIN 0.0172 for int == int 0.0252 for EqDefault<int>.Equals 1.4651162790697674 factor slowdown for EqDefault<int>.Equals

Analyzing Test Results

First consider relative time. In the first three testing scenarios, the slowdown was approximately by a factor of 100, or two orders of magnitude. In the last test, the slowdown was approximately by a factor of 1000, or three orders of magnitude.

Now consider absolute time. In the last test, the first call to EqDefault<int>.Equals(0, 1) took 106831 milliseconds, which is 106 seconds, which is more than 1.5 minutes. That is incredibly painful. That one call increases are startup time by over 50%.

Workaround

Luckily there is an easy workaround for us. We replaced calls like ma == mb with

ma.Map(a => mb.Map(b => a == b).IfNone(false)).IfNone(mb.IsNone).

This wasn’t too much work for us because Visual Studio allows us to search for symbolic references to the that operator, and we only had about a dozen such calls.

Requested Fix

For a proper fix, can the optimization of EqDefault<>.Equals be improved?

[TysonMN]

It is ok with me that it is not high on your list. I am in no rush either. We used the workaround described above to avoid the slow down. I created this issue to make sure you knew the performance cost of this code and so someone else with this problem might come across this issue.

[louthy]

One quick win might be to use Seq rather than Lst