Support AutoMapper's ProjectTo in DataSourceLoader

As described in https://www.devexpress.com/Support/Center/Question/Details/T758528/modify-the-datasourceloader-to-support-projection-as-part-of-the-original-query-operation and referenced threads - repeated below for simplicity

In a nutshell, the issue is this;

It is best practice in EF to return a DTO rather than the original object. Regardless of best practice, efficiency demands in so in my application as I have tables with large text fields that are not necessary for populating lists and would increase the size of the payload over 100x. I get the data for my lists using DataSourceLoader GET controllers, and I use filtering, sorting and grouping in the DataSourceLoader extensively. I ProjectTo to ensure that my payload from SQL to API, and my payload from API to client are efficient and contain no more data that is necessary.

At the moment, it is impossible to perform operations on the full set of object properties, but return only a subset using ProjectTo. Any property specified in the options e.g. a filter occurs after the ProjectTo, so the property is not available for filtering at that point in the SQL. As per the ticket, you cannot simply operate on the data after it is returned, as it breaks other elements of the returned set for more complex operations like grouping.

Also, a Select is not the answer as this requires far too much hard coding to move between types - this is what automapper and ProjectTo are for.

At the moment I have created a workaround that;

1. Returns the DataSourceLoader result if a Select is specified (obviously no projection is required in this case)
2. Programatically identifies the key of the original entity (e.g. User => UserId)
3. Runs the DataSourceLoader without Projecting, but returning only the Id of the entity - at this point I have all of the IDs matching the original query - IDset
4. Performs a simple where(x=>IDset.Contains(x=>[IDProperty])).ProjectTo<UserDto>()

This works, but it would be far better if the datasourceloader could be modified to append my projection so it occurs after the datasourceloader filtering / sorting / grouping. I can’t see that this would require much modification.

My code below for anyone else with this issue.

var src = _context.Approval.Where(x =>
  x.ProjectId == _userService.Project_ID &&
  x.PublishDate != null &&
  x.NewApprovalId == null).Include(x => x.ApprovalTo);
var result = _context.FilterAsDto<Approval, ApprovalListDto>(src, loadOptions);

public LoadResult FilterAsDto<T, TDto>(Func<T, bool> preFilter, DataSourceLoadOptions loadOptions) where T : class
{
	var qryResult = DataSourceLoader.Load(Set<T>().Where(preFilter), loadOptions);
	if (loadOptions.Select == null || loadOptions.Select.Count()==0) return FilterAsDto<T, TDto>(qryResult, loadOptions);
	else return qryResult;
	
}

public LoadResult FilterAsDto<T, TDto>(IQueryable<T> sourceQuery, DataSourceLoadOptions loadOptions) where T : class
{

	var qryResult = DataSourceLoader.Load(sourceQuery, loadOptions);
	if (loadOptions.Select == null || loadOptions.Select.Count() == 0) return FilterAsDto<T, TDto>(qryResult, loadOptions);
	else return qryResult;
}

private LoadResult FilterAsDto<T, TDto>(LoadResult loadedData, DataSourceLoadOptions loadOptions) where T : class
{
	var pkey = Model.FindEntityType(typeof(T)).FindPrimaryKey().Properties.Select(n => n.Name).Single();
	var pKeyExp = Expression.Parameter(typeof(T));
	var pKeyProperty = Expression.PropertyOrField(pKeyExp, pkey);
	var keySelector = Expression.Lambda<Func<T, int>>(pKeyProperty, pKeyExp).Compile();

	if (loadedData.data is IEnumerable<Group>) return loadedData;
	else
	{
		var OriginalSummary = loadedData.summary;
		List<int> idList = loadedData.data.Cast<T>().Select(keySelector).ToList();

		var pKeyExpDto = Expression.Parameter(typeof(TDto));
		var pKeyPropertyDto = Expression.PropertyOrField(pKeyExpDto, pkey);
		var method = idList.GetType().GetMethod("Contains");
		var call = Expression.Call(Expression.Constant(idList), method, pKeyPropertyDto);
		var lambda = Expression.Lambda<Func<TDto, bool>>(call, pKeyExpDto);
		var defOptions = new DataSourceLoadOptionsBase();
		defOptions.Sort = loadOptions.Sort;
		defOptions.RequireTotalCount = loadOptions.RequireTotalCount;
		var returnData= DataSourceLoader.Load(Set<T>().ProjectTo<TDto>(_mapper.ConfigurationProvider).Where(lambda), defOptions);

		returnData.summary = OriginalSummary;
		returnData.totalCount = loadedData.totalCount;

		return returnData;
	}
}

[Arafel-BR]

I also have need for this implementation. While our development team managed to use ProjectTo to with the DataSourceLoader.Load() method to list data for the grid we can’t filter nor order by many of the columns.

[statler]

Thanks Aleksey

1. AutoMapperProfileService and AutoMapperProfileORM are just my automapper configurations. These are a standard way of creating the maps for automapper. Example below

2. VisitAndConvert is a method of the ExpressionVisitor. Code for the ParameterVisitor which inherits this class, and overrides VisitParameter follows.

3. I don’t mind how it is delivered. My interest is in having this code available for others as it has taken ages to figure it out and hopefully now no-one else has to. This all works fine in my code, and I am on a deadline for the foreseeable future, so it is unlikely I will have time to create the project or manage it - I am hoping this is something someone in Devex finds valuable enough to manage?

    using System;
    using System.Collections.ObjectModel;
    using System.Linq.Expressions;
   
    namespace cpDataASP.Helpers
    {
    	public class ParameterVisitor : ExpressionVisitor
    	{
    		private readonly ReadOnlyCollection<ParameterExpression> _from;
    		private readonly ParameterExpression _to;
    		public ParameterVisitor(
    			ReadOnlyCollection<ParameterExpression> from,
    			ParameterExpression to)
    		{
    			if (from == null) throw new ArgumentNullException("from");
    			if (to == null) throw new ArgumentNullException("to");
    			this._from = from;
    			this._to = to;
    		}
    		protected override Expression VisitParameter(ParameterExpression node)
    		{
    			for (int i = 0; i < _from.Count; i++)
    			{
    				if (node == _from[i]) return _to;
    			}
    			return node;
    		}
    	}
    }
   

Example automapper code

	using AutoMapper;
	using cpDataASP.ControllerModels;
	using cpDataORM.Models;

	namespace cpDataASP.Helpers
	{
		public class AutoMapperProfileService : Profile
		{
			public AutoMapperProfileService()
			{ 
				CreateMap<LotImportDto, Lot>();
				CreateMap<ContractNotice, ContractNoticeListDto>()
					.ForMember(dest => dest.RequestByName, opt => opt.MapFrom(src => src.RequestBy == null ? "" : src.RequestBy.FirstName + " " + src.RequestBy.LastName)).IncludeAllDerived()
					.ForMember(dest => dest.CnToIDs, opt => opt.MapFrom(src => src.CnTos.Where(x => x.NoticeToId != null).Select(x => x.NoticeToId.Value).ToList())).IncludeAllDerived()
					.ForMember(dest => dest.CnToNames, opt => opt.MapFrom(src => src.CnTos.Select(x => x.NoticeTo == null ? x.NoticeEmail : x.NoticeTo.FirstName + " " + x.NoticeTo.LastName).ToList())).IncludeAllDerived()
					.ForMember(dest => dest.NumberOfResponses, opt => opt.MapFrom(src => src.CnResponses.Count())).IncludeAllDerived()
					.ForMember(dest => dest.NumberOfActionedResponses, opt => opt.MapFrom(src => src.CnResponses.Count(x => x.DateActioned != null))).IncludeAllDerived();

			}
		}
	}