[BUG] Complex queries on nested documents only ever return first result set, even after subsequent queries with different criteria

Version Which LiteDB version/OS/.NET framework version are you using. 5.0.9/Windows 10/.NET Framework 4.8

Describe the bug Complex queries on nested documents only ever return first result set, even after subsequent queries with different criteria.

I was experiencing the issue in my project, and decided to distill it down to the minimum code needed to reproduce. It is pasted below. In the PerformTest() method, the second test fails. It returns the result of the first query always. (Even if I close and re-open the database.)

Code to Reproduce

public class Parent
{
	[BsonId]
	public int Id { get; set; }

	[BsonRef("child")]
	public List<Child> Children { get; set; }
}

public class Child
{
	[BsonId]
	public int Id { get; set; }

	public string Name { get; set; }
}

void Main()
{
	ResetTest();
	InitializeTest();
	PerformTest();
}

private static string DatabasePath = Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.Desktop), "Test.db");

private void ResetTest()
{
	string databasePath = Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.Desktop), "Test.db");
	if (File.Exists(databasePath))
	{
		File.Delete(databasePath);
	}
}

private void InitializeTest()
{
	LiteDatabase database = new LiteDatabase(DatabasePath);
	ILiteCollection<Parent> parentCollection = database.GetCollection<Parent>("parent");
	ILiteCollection<Child> childCollection = database.GetCollection<Child>("child");

	Child firstChild = new Child { Name = "TestA" };
	Child secondChild = new Child { Name = "TestB" };
	Child thirdChild = new Child { Name = "Third" };

	childCollection.Insert(firstChild);
	childCollection.Insert(secondChild);
	childCollection.Insert(thirdChild);

	Parent firstParent = new Parent { Id = 1, Children = new List<Child> { firstChild, secondChild } };
	Parent secondParent = new Parent { Id = 2, Children = new List<Child> { thirdChild } };

	parentCollection.Insert(firstParent);
	parentCollection.Insert(secondParent);
	
	database.Dispose();
}

private void PerformTest()
{
	LiteDatabase database = new LiteDatabase(DatabasePath);
	ILiteCollection<Parent> parentCollection = database.GetCollection<Parent>("parent").Include(p => p.Children);

	ILiteQueryable<Parent> query = parentCollection.Query().Where(p => p.Children.Where(c => c.Name.Contains("Test")).Any());
	Debug.Assert(query.First().Id == 1, $"Did not find correct ID of parent where at least one child's name contains 'Test'. ID is {query.First().Id}.");
	
	query = parentCollection.Query().Where(p => p.Children.Where(c => c.Name.Contains("Third")).Any());
	Debug.Assert(query.First().Id == 2, $"Did not find correct ID of parent where at least one child's name contains 'Third'. ID is {query.First().Id}.");
	
	database.Dispose();
}

Output

Fail: Did not find correct ID of parent where at least one child's name contains 'Third'. ID is 1.

Expected behavior Each query returns the correct results, as a direct query via the Studio application does. Specifically, the second test should pass, where the resulting object’s ID is 2, because the second Parent contains the Child with “Third” in the Name.

Screenshots/Stacktrace Inspecting the ILiteQueryable object, I can see that the resulting queries will look something like this.

• First test: SELECT $ from parent INCLUDE $.Children WHERE COUNT(FILTER($.Children[*]=>@.Name LIKE '%Test%'))>0
• Second test: SELECT $ from parent INCLUDE $.Children WHERE COUNT(FILTER($.Children[*]=>@.Name LIKE '%Third%'))>0

Running these queries through the Studio application returns the correct results. But the same queries as generated and run via the C# API do not.

Additional context N/A

[RobbieLD]

@AlBannaTechno Thanks again for responding! I think you have solved the issue! Your test script produces the correct results. Notice that is has a key difference from my original code.

Your two select statements are as follows:

var result2 = parentCollection.Include(p => p.Children).Query().Where(p => p.Children.Select(c => c.Name).Any(n => n.Contains("rd"))).ToList();
var result3 = parentCollection.Include(p => p.Children).Query().Where(p => p.Children.Select(c => c.Name).Any(n => n.Contains("Test"))).ToList();

Mine were as follows:

var result2 = parentCollection.Include(p => p.Children).Query().Where(p => p.Children.Where(c => c.Name.Contains("rd")).Any()).ToList();
var result3 = parentCollection.Include(p => p.Children).Query().Where(p => p.Children.Where(c => c.Name.Contains("Test")).Any()).ToList();

Yours produces the correct results, while mine always gives the same (first) result for every query, and continues to fail the test.

It looks like the key is to manually select every property that you want to filter on, rather than trying to dig into multiple layers (the c.Name.Contains is what hurt me, apparently).

I should be able to use your solution in my project, so thank you very much!

I think I would still like to keep this issue open, since something that seems like it should work does not. Either it should be unsupported syntax, or it should be fixed. Note that there is already precedence for unsupported syntax. For example, if I change my query to combine the Where and the Any, like this:

result2 = parentCollection.Include(p => p.Children).Query().Where(p => p.Children.Any(c => c.Name.Contains("rd"))).ToList();

I get the following exception from LiteDB.

System.NotSupportedException : Any/All requires simple parameter on left side. Eg: `x.Customers.Select(c => c.Name).Any(n => n.StartsWith('J'))`

So perhaps they could use a similar exception for my scenario, if it truly is not supported. Do you agree?

Thanks again for all your help!

Just adding my own experience here. This really does seem to be a bug but your solution of selecting the properties (in my case two of them) in the child object and then performing an Any on them seems to have done the trick. I suspect this might have something to do with the fact that normally you’d just call .Any() with the expression you want but as was pointed out earlier in this thread that throws a not supported exception so calling .Where() with the expression and then .Any() on that seemed to be a sneaky work around but ultimately one that bit us for trying to be too clever.

[micahmo]

LiteDB does not support everything like LINQ

Yes, I am discovering that! However, as you found, the syntax I tried to use seemed to be supported. I even inspected the ILiteQueryable and I could see that the query was valid/correct. As you said, it seems like there may be a caching issue. I’m glad you could reproduce! Thanks for tagging the core team, and thanks again for your help.