"Collection was modified; enumeration operation may not execute" thrown by await FastTextLanguageDetector.FromStoreAsync in .NET Core 3.1

The following code throws an InvalidOperationException with the message “Collection was modified; enumeration operation may not execute.” on the line that calls await FastTextLanguageDetector.FromStoreAsync when the application targets .NET Core 3.1.

However, it works fine when targeting .NET 5!

using System;
using System.IO;
using System.Threading.Tasks;
using Catalyst;
using Catalyst.Models;
using Mosaik.Core;
using Version = Mosaik.Core.Version;

namespace CatalystSimilarityExample
{
    class Program
    {
        static async Task Main()
        {
            const string modelFolderName = "catalyst-models";
            Storage.Current = new OnlineRepositoryStorage(new DiskStorage(modelFolderName));
            var languageDetector = await FastTextLanguageDetector.FromStoreAsync(
                Language.Any,
                Version.Latest,
                ""
            );
        }
    }
}

The stack trace shows this:

at System.ThrowHelper.ThrowInvalidOperationException_InvalidOperation_EnumFailedVersion()
at System.Collections.Generic.Dictionary`2.KeyCollection.Enumerator.MoveNext()
at Catalyst.Models.FastText.CompactSupervisedModel()
at Catalyst.Models.FastTextLanguageDetector.<FromStoreAsync>d__5.MoveNext()
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
at CatalystSimilarityExample.Program.<Main>d__0.MoveNext() in C:\\Users\\Dan\\source\\repos\\ParallelLinqExample\\CatalystSimilarityExample\\Program.cs:line 17

[gillonba]

Also, in my testing of the same code the LanguageDetector was only 56.8% accurate. For example code, not good! I run the samples to decide whether I want to use the library for real and that is somewhat less than inspiring. Maybe I should just bite the bullet call spaCy scripts from my application? It would be much better to use pure .Net but only if it works. Maybe I need to look into upgrading to .Net 5?

[ProductiveRage]

@ProductiveRage fixed the bug with loading the FT model - was a recent memory optimization added to the FT model that broke loading classifier models from disk.

Sorry, @theolivenbaum, I missed this update somehow - can confirm that I’ve tested with 3.1 and it works fine now! I’m closing this issue even though there seems to be a question from @gillonba about detection accuracy… I suspect that that should be a separate issue?