[BUG] Corrupt / damaged / unreadable records in table

Version LiteDB v5.0.7 .NET Framework 4.8 Windows

Describe the bug

We have encountered a strange situation where it would appear that part of the one table in our LiteDB has become corrupt/unreadable for reasons unknown.

Our database has 7 tables, 6 can be read and written to fine, the other 1 cannot be read, it however seems to be able to be written to just fine.

From our application logs, I know that we should have 14 records in the table, however we are unable to read any data from it, via our own applications and via the LiteDB Studio (v1.0.1.0). Both throw the same exception as below.

I was able to export data to JSON from LiteDB Studio, however there were only 4 records exported and as I said, there should be 14.

Is there a way this data can be recovered/repaired?

Do you know what may have caused this exception and therefore loss of data if it is not recoverable?

Expected behavior Return all data in the table

Screenshots/Stacktrace

System.NotSupportedException: BSON type not supported
   at LiteDB.Engine.BufferReader.ReadElement(HashSet`1 remaining, String& name)
   at LiteDB.Engine.BufferReader.ReadDocument(HashSet`1 fields)
   at LiteDB.Engine.BufferReader.ReadElement(HashSet`1 remaining, String& name)
   at LiteDB.Engine.BufferReader.ReadDocument(HashSet`1 fields)
   at LiteDB.Engine.BufferReader.ReadElement(HashSet`1 remaining, String& name)
   at LiteDB.Engine.BufferReader.ReadArray()
   at LiteDB.Engine.BufferReader.ReadElement(HashSet`1 remaining, String& name)
   at LiteDB.Engine.BufferReader.ReadDocument(HashSet`1 fields)
   at LiteDB.Engine.DatafileLookup.Load(PageAddress rawId)
   at LiteDB.Engine.DatafileLookup.Load(IndexNode node)
   at LiteDB.Engine.BasePipe.<LoadDocument>d__6.MoveNext()
   at LiteDB.Engine.BasePipe.<Filter>d__8.MoveNext()
   at LiteDB.Engine.QueryPipe.<Select>d__2.MoveNext()
   at LiteDB.Engine.QueryExecutor.<>c__DisplayClass10_0.<<ExecuteQuery>g__RunQuery|0>d.MoveNext()
   at LiteDB.BsonDataReader..ctor(IEnumerable`1 values, String collection)
   at LiteDB.Engine.QueryExecutor.ExecuteQuery(Boolean executionPlan)
   at LiteDB.Engine.QueryExecutor.ExecuteQuery()
   at LiteDB.Engine.LiteEngine.Query(String collection, Query query)
   at LiteDB.SharedEngine.Query(String collection, Query query)
   at LiteDB.LiteQueryable`1.<ToDocuments>d__26.MoveNext()
   at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
   at System.Linq.Buffer`1..ctor(IEnumerable`1 source)
   at System.Linq.OrderedEnumerable`1.<GetEnumerator>d__1.MoveNext()
   at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
   at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)

[sqbitz]

Here is a copy of the database we’re using. I removed all the collections except for where the problem actually lies (ProjectItem). Application.zip

Just do a SELECT $ FROM ProjectItem; and you’ll get the BSON type not supported error. I really hope this issue can be fixed. We see this issue crop up around once a month randomly and it completely screws up our data.

[sqbitz]

We are seeing this in one of our LiteDB applications too. We are also seeing the issue around nullable DateTime fields if that helps. I think I’ve got a corrupted database backed up somewhere. I’ll upload it as soon as I find it.