LINQ query parsing regression in 3.0.0-rc.1

Certain combination of LINQ queries seem to be causing a parsing failure in 3.0.0-rc.1. The query works fine in 2.9.8. As you can see below, linq2db is taking a local variable name (searchDateFrom) and trying to create a member access expression on a database class (Order) which of course does not contain this field. I’ve attempted to strip down the original query as much as possible to simplify the problem. I’ve also included a link to the open-source project where you can see how this query was originally implemented.

Exception message:
  Field 'TestLinq2Db.App+<>c__DisplayClass4_0.searchDateFrom' is not defined for type 'TestLinq2Db.Order'
Stack trace:
   at System.Linq.Expressions.Expression.Field(Expression expression, FieldInfo field)
   at System.Linq.Expressions.Expression.MakeMemberAccess(Expression expression, MemberInfo member)
   at LinqToDB.Expressions.Extensions.TransformX(MemberExpression e, Func`2 func)
   at LinqToDB.Expressions.Extensions.Transform(Expression expr, Func`2 func)
   at LinqToDB.Linq.Builder.SelectContext.GetExpression(Expression expression, Expression levelExpression, Expression memberExpression)
   at LinqToDB.Linq.Builder.SelectContext.ProcessScalar[T](Expression expression, Int32 level, Func`4 action, Func`1 defaultAction, Boolean throwOnError)
   at LinqToDB.Linq.Builder.SelectContext.IsExpressionInternal(Expression expression, Int32 level, RequestFor requestFlag)
   at LinqToDB.Linq.Builder.SelectContext.IsExpression(Expression expression, Int32 level, RequestFor requestFlag)
   at LinqToDB.Linq.Builder.ExpressionBuilder.ConvertCompare(IBuildContext context, ExpressionType nodeType, Expression left, Expression right)
   at LinqToDB.Linq.Builder.ExpressionBuilder.ConvertPredicate(IBuildContext context, Expression expression)
   at LinqToDB.Linq.Builder.ExpressionBuilder.BuildSearchCondition(IBuildContext context, Expression expression, List`1 conditions, Boolean isNotExpression)
   at LinqToDB.Linq.Builder.ExpressionBuilder.BuildWhere(IBuildContext parent, IBuildContext sequence, LambdaExpression condition, Boolean checkForSubQuery, Boolean enforceHaving)
   at LinqToDB.Linq.Builder.WhereBuilder.BuildMethodCall(ExpressionBuilder builder, MethodCallExpression methodCall, BuildInfo buildInfo)
   at LinqToDB.Linq.Builder.MethodCallBuilder.BuildSequence(ExpressionBuilder builder, BuildInfo buildInfo)
   at LinqToDB.Linq.Builder.ExpressionBuilder.BuildSequence(BuildInfo buildInfo)
   at LinqToDB.Linq.Builder.SelectBuilder.BuildMethodCall(ExpressionBuilder builder, MethodCallExpression methodCall, BuildInfo buildInfo)
   at LinqToDB.Linq.Builder.MethodCallBuilder.BuildSequence(ExpressionBuilder builder, BuildInfo buildInfo)
   at LinqToDB.Linq.Builder.ExpressionBuilder.BuildSequence(BuildInfo buildInfo)
   at LinqToDB.Linq.Builder.GroupByBuilder.BuildMethodCall(ExpressionBuilder builder, MethodCallExpression methodCall, BuildInfo buildInfo)
   at LinqToDB.Linq.Builder.MethodCallBuilder.BuildSequence(ExpressionBuilder builder, BuildInfo buildInfo)
   at LinqToDB.Linq.Builder.ExpressionBuilder.BuildSequence(BuildInfo buildInfo)
   at LinqToDB.Linq.Builder.SelectBuilder.BuildMethodCall(ExpressionBuilder builder, MethodCallExpression methodCall, BuildInfo buildInfo)
   at LinqToDB.Linq.Builder.MethodCallBuilder.BuildSequence(ExpressionBuilder builder, BuildInfo buildInfo)
   at LinqToDB.Linq.Builder.ExpressionBuilder.BuildSequence(BuildInfo buildInfo)
   at LinqToDB.Linq.Builder.ExpressionBuilder.Build[T]()
   at LinqToDB.Linq.Query`1.CreateQuery(IDataContext dataContext, Expression expr)
   at LinqToDB.Linq.Query`1.GetQuery(IDataContext dataContext, Expression& expr)
   at LinqToDB.Linq.ExpressionQuery`1.GetQuery(Expression& expression, Boolean cache)
   at LinqToDB.Linq.ExpressionQuery`1.<GetForEachAsync>d__21.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.ConfiguredTaskAwaitable.ConfiguredTaskAwaiter.GetResult()
   at LinqToDB.AsyncExtensions.<ToListAsync>d__6`1.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
   at TestLinq2Db.App.<TestFailedQuery>d__4.MoveNext() in C:\Users\Shane\source\repos\ZboxInternalTools\TestLinq2Db\App.cs:line 82

Steps to reproduce

        public async Task TestFailedQuery()
        {
            var provider = new SqlServerDataProvider(ProviderName.SqlServer, SqlServerVersion.v2017);
            var db = new DataContext(provider, Configuration.GetConnectionString("testdb"));
            var orders = db.GetTable<Order>();
            var addresses = db.GetTable<Address>();
            var searchDateFrom = DateTime.Parse("6/25/2020 4:00am");

            IQueryable<Order> query = orders;

            query =
                from o in query
                join oba in addresses on o.BillingAddressId equals oba.Id
                // note: original query had where clauses written here
                select o;

            // note: if this where clause is moved before the prior statement, the exception does not occur
            query = query.Where(o => searchDateFrom <= o.CreatedOnUtc);

            var finalQuery =
                from o in query
                group o by 1
                into result
                select new {
                    OrderCount = result.Count()
                };

            var retList = await finalQuery.ToListAsync(); //throws exception
        }
    class Order
    {
        public int Id { get; set; }
        public int BillingAddressId { get; set; }
        public DateTime CreatedOnUtc { get; set; }
    }
    class Address
    {
        public int Id { get; set; }
    }

You can also see this query in its original context here: https://github.com/nopSolutions/nopCommerce/blob/33ff21b9f70226e926ffc141fb3512b2bd7d9535/src/Libraries/Nop.Services/Orders/OrderReportService.cs#L157..L248

Environment details

linq2db version: 3.0.0-rc.1 Database Server: Azure SQL Server Database Provider: SQL Server v2017 Operating system: Win 10 Pro .NET Framework: .Net Core 3.1

[sdanyliv]

@Shane32, thanks for investigation. I’m aware how to fix that. Problem that IsExpression should never throw exception but instead that it should return false. Will try to find all places which may cause such error.

[Shane32]

The regression was introduced in PR #2273 (new feature: Value Converters)

This code was added to ExpressionBuilder.SqlBuilder.ConvertCompare:

			ISqlExpression l;
			ISqlExpression r;
			if (context?.IsExpression(left, 0, RequestFor.Field).Result == true)
			{
				l = ConvertToSql(context, left);
				r = ConvertToSql(context, right, true, QueryHelper.GetColumnDescriptor(l));
			}
			else
			{
				r = ConvertToSql(context, right, true);
				l = ConvertToSql(context, left, false, QueryHelper.GetColumnDescriptor(r));
			}

The error stems from the IsExpression test. As the IsExpression test only processes the left side of the argument, the problem only occurs when the left operand is not a property of the model. However, the bug may not be within that code, but deeper in. Perhaps where SelectContext.ProcessScalar calls GetExpression here (since it passes in Body which is not likely correct):

				if (root.NodeType == ExpressionType.Parameter)
				{
					var levelExpression = expression.GetLevelExpression(Builder.MappingSchema, level - 1);
					var newExpression   = GetExpression(expression, levelExpression, Body);

					Builder.UpdateConvertedExpression(expression, newExpression);

That’s about as much as I can figure out. I hope it helps.