Potential bug when using (I)LIKE with array columns

Dear Postgres team,

I created a repo with a minimal example to reproduce our issue with ILike. See the readme in the repo for a detailed description.

In short: We have array columns (by intention) to store e.g. a user’s competencies, topics etc. Now, users should be able to search across these columns, also with partial wording. Let’s say, user A stores ["Artificial Intelligence", "Big Data", "Robotics"]. Then, any user should be able to find this entry by search for "intelli".

Regarding to the “Array Type Mapping” documentation, we assumed that we could archive this by:

.Where(n => n.Topics.Any(s => EF.Functions.ILike(s, $"%{searchTerm1}%")))

It throws an NullReferenceException:

System.NullReferenceException: Object reference not set to an instance of an object.
   at Npgsql.EntityFrameworkCore.PostgreSQL.Query.NpgsqlSqlExpressionFactory.ApplyTypeMappingsOnItemAndArray(SqlExpression itemExpression, SqlExpression arrayExpression)
   at Npgsql.EntityFrameworkCore.PostgreSQL.Query.NpgsqlSqlExpressionFactory.ApplyTypeMappingOnAny(PostgresAnyExpression postgresAnyExpression)
   at Npgsql.EntityFrameworkCore.PostgreSQL.Query.NpgsqlSqlExpressionFactory.ApplyTypeMapping(SqlExpression sqlExpression, RelationalTypeMapping typeMapping)
   at Microsoft.EntityFrameworkCore.Query.SqlExpressionFactory.ApplyDefaultTypeMapping(SqlExpression sqlExpression)
   at Npgsql.EntityFrameworkCore.PostgreSQL.Query.NpgsqlSqlExpressionFactory.Any(SqlExpression item, SqlExpression array, PostgresAnyOperatorType operatorType)
   at Npgsql.EntityFrameworkCore.PostgreSQL.Query.Internal.NpgsqlSqlTranslatingExpressionVisitor.VisitArrayMethodCall(MethodInfo method, ReadOnlyCollection`1 arguments)
   at Npgsql.EntityFrameworkCore.PostgreSQL.Query.Internal.NpgsqlSqlTranslatingExpressionVisitor.VisitMethodCall(MethodCallExpression methodCall)
   at System.Linq.Expressions.MethodCallExpression.Accept(ExpressionVisitor visitor)
   at System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)
   at Microsoft.EntityFrameworkCore.Query.RelationalSqlTranslatingExpressionVisitor.TranslateInternal(Expression expression)
   at Microsoft.EntityFrameworkCore.Query.RelationalSqlTranslatingExpressionVisitor.Translate(Expression expression)
   at Microsoft.EntityFrameworkCore.Query.RelationalQueryableMethodTranslatingExpressionVisitor.TranslateExpression(Expression expression)
   at Microsoft.EntityFrameworkCore.Query.RelationalQueryableMethodTranslatingExpressionVisitor.TranslateLambdaExpression(ShapedQueryExpression shapedQueryExpression, LambdaExpressi
on lambdaExpression)
   at Microsoft.EntityFrameworkCore.Query.RelationalQueryableMethodTranslatingExpressionVisitor.TranslateWhere(ShapedQueryExpression source, LambdaExpression predicate)
   at Microsoft.EntityFrameworkCore.Query.QueryableMethodTranslatingExpressionVisitor.VisitMethodCall(MethodCallExpression methodCallExpression)
   at System.Linq.Expressions.MethodCallExpression.Accept(ExpressionVisitor visitor)
   at System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)
   at Microsoft.EntityFrameworkCore.Query.QueryCompilationContext.CreateQueryExecutor[TResult](Expression query)
   at Microsoft.EntityFrameworkCore.Storage.Database.CompileQuery[TResult](Expression query, Boolean async)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.CompileQueryCore[TResult](IDatabase database, Expression query, IModel model, Boolean async)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.<>c__DisplayClass9_0`1.<Execute>b__0()
   at Microsoft.EntityFrameworkCore.Query.Internal.CompiledQueryCache.GetOrAddQuery[TResult](Object cacheKey, Func`1 compiler)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.Execute[TResult](Expression query)
   at Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryProvider.Execute[TResult](Expression expression)
   at Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryable`1.GetEnumerator()
   at BugILikePostgres.Program.Main(String[] args) in C:\_Data\Repositories\BugILikePostgres\BugILikePostgres\Program.cs:line 73

When I exchanged the matchExpress with the pattern parameter, it “works” . But obviously, we cannot use pattern anymore in such a case. But this exchanging shows, that there is no null field.

Is this a bug, or have we misunderstood the documentation? 🙂

For reference: I mentioned this issue first on issue #395.

[JohnDitno]

@roji from further reading online I don’t believe unnest can use indexes so your right performance is likely to be poor.

I did find an extension called parray_gin (https://github.com/theirix/parray_gin/) which seems to provide GIN index and operator support for arrays with partial match but this is likely to fall outside of EFCore.PG support.

I think to best option to support this use case would be to normalise the array column into a separate table so LIKE operations can leverage the indexing for better performance

[roji]

Thanks for looking into this further.

FWIW I’m open to any translation that makes sense - including those which are complicated and/or require extensions (although exotic extensions are maybe a bit too much). What I’m wary of, is implementing translations which definitely yield bad performance in most/all cases where they’re used, and where this isn’t very obvious to the person writing the LINQ query.