Calling .AsQueryable() on List navigation properties with subqueries generates invalid SQL

Steps to reproduce

GitHub repo: https://github.com/JulianKlauser/EF.Core.Subqueries (adjust connection string to local db).

The issue

Subqueries on navigation properties currently do not work as expected. If an .AsQueryable() is called on a List navigation property an exception is thrown (the generated SQL is not valid for the MySql Server).

This works:

context.Set<ParentEntity>()   
   .Where(parent => parent.ChildEntities
   .Any(child => child.Value =="Dummy")).ToList();

This does not:

context.Set<ParentEntity>()
   .Where(parent => parent.ChildEntities.AsQueryable()
   .Any(child => child.Value == "Dummy")).ToList();

The exception:

MySql.Data.MySqlClient.MySqlException
  HResult=0x80004005
  Message=Unknown column 'parent.Id' in 'where clause'
  Source=MySqlConnector
  StackTrace:
   at MySql.Data.MySqlClient.MySqlDataReader.ActivateResultSet(ResultSet resultSet) in C:\projects\mysqlconnector\src\MySqlConnector\MySql.Data.MySqlClient\MySqlDataReader.cs:line 98
   at MySql.Data.MySqlClient.MySqlDataReader.<ReadFirstResultSetAsync>d__94.MoveNext() in C:\projects\mysqlconnector\src\MySqlConnector\MySql.Data.MySqlClient\MySqlDataReader.cs:line 307
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at MySql.Data.MySqlClient.MySqlDataReader.<CreateAsync>d__93.MoveNext() in C:\projects\mysqlconnector\src\MySqlConnector\MySql.Data.MySqlClient\MySqlDataReader.cs:line 292
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at MySqlConnector.Core.TextCommandExecutor.<ExecuteReaderAsync>d__1.MoveNext() in C:\projects\mysqlconnector\src\MySqlConnector\Core\TextCommandExecutor.cs:line 37
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at MySql.Data.MySqlClient.MySqlCommand.ExecuteDbDataReader(CommandBehavior behavior) in C:\projects\mysqlconnector\src\MySqlConnector\MySql.Data.MySqlClient\MySqlCommand.cs:line 267
   at Microsoft.EntityFrameworkCore.Storage.Internal.RelationalCommand.Execute(IRelationalConnection connection, DbCommandMethod executeMethod, IReadOnlyDictionary`2 parameterValues)
   at Microsoft.EntityFrameworkCore.Storage.Internal.RelationalCommand.ExecuteReader(IRelationalConnection connection, IReadOnlyDictionary`2 parameterValues)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryingEnumerable`1.Enumerator.BufferlessMoveNext(DbContext _, Boolean buffer)
   at Pomelo.EntityFrameworkCore.MySql.Storage.Internal.MySqlExecutionStrategy.Execute[TState,TResult](TState state, Func`3 operation, Func`3 verifySucceeded)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryingEnumerable`1.Enumerator.MoveNext()
   at Microsoft.EntityFrameworkCore.Query.Internal.LinqOperatorProvider.<_TrackEntities>d__17`2.MoveNext()
   at Microsoft.EntityFrameworkCore.Query.Internal.LinqOperatorProvider.ExceptionInterceptor`1.EnumeratorExceptionInterceptor.MoveNext()
   at System.Collections.Generic.List`1.AddEnumerable(IEnumerable`1 enumerable)
   at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
   at EF.Core.Subqueries.Program.Main(String[] args) in D:\Git\EF.Core.Subqueries\EF.Core.Subqueries\Program.cs:line 22

Inner Exception 1:
MySqlException: Unknown column 'parent.Id' in 'where clause'

This is the generated (invalid) SQL from the logger:

SELECT `parent`.`Id`, `parent`.`Value`
      FROM `parent` AS `parent`
      WHERE EXISTS (
           SELECT 1
           FROM (
               SELECT `c`.`Id`, `c`.`ParentEntityId`, `c`.`Value`
               FROM `child` AS `c`
               WHERE `parent`.`Id` = `c`.`ParentEntityId`
          ) AS `t`
          WHERE `t`.`Value` = 'Dummy')

The reason why we need the .AsQueryable() is that we want to build the child expression dynamically and pass it in as a Expression<Func<ChildEntity,bool>>.

Further technical details

MySQL version: MariaDB 10.2.17 (also tried on MariaDB 10.3.7) Operating system: Windows 10 Pro (version 1809) Pomelo.EntityFrameworkCore.MySql version: v2.2.0 Microsoft.AspNetCore.App version: v2.2.0

Other details about my project setup: Using MariaDb instead of MySql.

[smitpatel]

To fix “derived table” limitation, it would require special casing in MySQL provider since it is database specific limitation. For the behavioral difference in both queries simply because of AsQueryable is due to Relinq. Relinq treats AsQueryable as result operator and causes a subquery. This should go away in 3.0 and if the query without AsQueryable is working then it would work with it too.

[JulianKlauser]

I see. Thank you for the insightful comments, I have a better grasp of what is happening. We’ll wait for 3.0 then.