[BUG] LiteDatabase is not thread safe

Version LiteDB 5.0.2 with .NET Core 3.1 (don’t think this part matters)

Describe the bug Per the suggestion at https://github.com/mbdavid/LiteDB/wiki/Concurrency, I create a single LiteDatabase instance to share with multiple thread. The code snippet inserts a single record then queue a Task to remove the record. Even though the deletion Tasks are running on a separate thread, the Tasks are chained so only one deletion is running at a time. At the end of the program, the code output 0 items. When using LiteDB Studio to view the database, there are still records in the table. The number of left-over records are random.

Code to Reproduce

class Program
    {
        
        private static LiteDatabase _db;
        private static Task _eventProcessingTask;
        private static readonly object _processingTaskLock = new object();
        private static string _tableName = "myTable";
        static void Main(string[] args)
        {
            _db = new LiteDatabase($"Filename=MyDb;Mode=Exclusive;Async=false;Flush=true;");
            
            for (int i = 1; i < 1000; i++)
            {
                Insert(i);
            }
            
            System.Threading.Thread.Sleep(5000);
            
            var table = _db.GetCollection<TableRow>(_tableName);
            Console.WriteLine($"There are {table.Count()} items.");
        }

        static void Insert(int id)
        {
            var table = _db.GetCollection<TableRow>(_tableName);
            table.Insert(new TableRow { Id = id, Data = $"item {id}"});
            
            lock (_processingTaskLock)
            {
                if (_eventProcessingTask == null)
                    _eventProcessingTask = Task.Run(() => Delete(id));
                else
                    _eventProcessingTask = _eventProcessingTask.ContinueWith((t) => Delete(id));
            }
        }

        static void Delete(int id)
        {
            var table = _db.GetCollection<TableRow>(_tableName);
            table.Delete(new BsonValue(id));
        }
    }

    public class TableRow
    {
        public int Id { get; set; }
        public string Data { get; set; }
    }

Expected behavior In-memory count should show the count of the items of the database in the file system.

[mbdavid]

Hi @Thdm1907, there is some points here:

Engine contains an async writer queue to write in disk (in async thread) all dirty pages direct to Log file. This process still occurs after user get “Insert OK” (or delete, update…). If you interrupt your program during this async write, this missing pages/transactions will not be write and when open database again you will see “half” operations.

When you call Dispose from LiteDatabase, this method waits for this async thread finish before close database, runs Checkpoint operation (coping LOG file into datafile) before everything finish. So, with no Dispose, you can get some “random” data inside database. Maybe is the case for you.

[pablopioli]

@pablopioli, can you open another issue about your error. Can be a concurrency problem but are not related with this issue.

Yes, sorry, it was late on night and wasn’t thinking properly. I’m building a repro app to attach to the issue.