Auto project columns based on `AsQueryable<T>` and `AsEnumerable<T>` generic type

The problem

Suppose I have an API and want to query the following Foo entity by its Name property:

public sealed class Foo : IEntityBase, IQueryableEntity
{
    public Foo()
    {
	Id = Guid.NewGuid();
	Name = string.Empty;
	Created = DateTimeOffset.UtcNow;
    }

    public Guid Id { get; init; }
    public string Name { get; init; }
    public DateTimeOffset Created { get; init; }
}

public interface IEntityBase
{
    Guid Id { get; }
    DateTimeOffset Created { get; }
}

public interface IQueryableEntity
{
    Guid Id { get; }
    string Name { get; }
}

This is what the request method would look like:

public async Task<IQueryableEntity[]> OnGetAsync(string query, CancellationToken cancellationToken)
{
    IQueryableEntity[] result = await context.Foos.AsQueryable<IQueryableEntity>()
	.Where(x => x.Name.Contains(query))
	.ToArrayAsync(cancellationToken);
    return result;
}

However the generated SQL includes all columns:

SELECT [c].[Id], [c].[Created], [c].[Name]
FROM [dbo].[Foos] AS [c]
WHERE (@__query_0 LIKE N'') OR CHARINDEX(@__query_0, [c].[Name]) > 0

The documentation says I should use .Select() to project only the necessary columns. So that would look like this:

    IQueryableEntity[] result = await context.Foos.AsQueryable<IQueryableEntity>()
	.Where(x => x.Name.Contains(query))
	.Select(x => new Foo
	{
	    Id = x.Id,
	    Name = x.Name
	}).ToArrayAsync(cancellationToken);

Although this does work, it no longer returns an IQueryableEntity array, instead it returns Foo[]. The problem with that is that if you return it, ASP.NET Core will include the Created property in the JSON result because it was initialized in the constructor, and this is Foo now, no longer IQueryableEntity.

Of course using an anonymous type in .Select() would fix this problem, but I can no longer work with the result as an IQueryableEntity and seems like there’s a lot more casting involved than need be (I could be and probably am wrong here).

The proposal

Take advantage of the AsQueryable<T> and AsEnumerable<T> and use T as the type for projection. This would probably be a breaking change if made default behavior, so instead introduce new extension methods for .Select():

public static IQueryable<T> Select<T>(this IQueryable<T> source);
public static IEnumerable<T> Select<T>(this IEnumerable<T> source);

[roji]

Hiding stuff behind a IBlogView may be enough to ensure client code is discouraged to do the wrong thing (e.g. accessing Posts collection), but when you try to serialize IBlogView[] that’s actually a Blog[] you won’t get the expected json serialized by asp.net core.

My point was that for serialization you can just use an anonymous type instead.

But yeah, if you’re writing a strongly-typed API whose results later get serialized, that’s true.

[roji]

A bit off topic maybe, but in order to better control serialization, I always copy what I read from the db to some sort of DTO. That’s certainly possible, but it effectively means never using EF’s change tracking capabilities. If you’re in a reading scenario, change tracking may actually be turned off on the whole context to improve perf.

It’s one thing to turn off change tracking in specific perf-sensitive areas, but quite another to systematically always project out to a DTO.

But I’m not disagreeing with the general concept… It’s true that LINQ tends to encourage over-fetching by getting all entity properties by default, requiring a specific gesture to selectively project only a subset. I agree that in various scenarios (especially disconnected ones where change tracking is frequently less relevant) systematic projection can be a good idea.

One part where I’m slightly less clear about, is the need for a separate DTO type - you can usually just project out the same type you’re reading from the database, but populating only the properties you need (you did this in your original post above):

return (IBlogView[])context.Blogs.Select(b => new Blog { Name = b.name, Rating = b.rating }).ToArray();

You’re still not exposing Blog directly - only through IBlogView which functions a bit like your DTO - but without having to copy data across (i.e. just expose a view rather than a copy). Of course, if you’re just sending data back via JSON (e.g. a typical web API), anonymous objects can make this even simpler, removing the need for even a view interface (since they’re serialized to JSON just as well, no need for a special named type).

To summarize, I definitely agree that it’s a good idea to carefully think about which properties you really need to get from the database and expose out. EF provides several techniques to do this (project out to the same type + view interface, project out to an anonymous type).

I agree it may be interesting think about making this even easier via a dedicated terminating operator effectively projects out to an untracked instance, populating only the properties which exist on an interface implemented by that type. I don’t think it makes sense for that operator to be simply called Select (since it has some specific semantics which Select does not have), and I also don’t think an enumerable version of that operator makes sense (it would rather be an EF-specific thing).