"ITable<T> considers only declared, not runtime type" or "How to work with abstract meta-table?"

May I start from the beginning? It can help to find the most suitable solution.

Origin of the problem

In my business model I have entities of two types (let’s say A and B). It’s convenient not to distinguish A and B in the part of business logic (sometimes A and B are equivalent from logic point of view: same business processes, same related data structures and so on). So I’m going to have (a part of) logic layer like this:

// project "Model.Interfaces"

public enum EntityType
{
	TypeA,
	TypeB,
}

public class Entity
{
	public Guid Id { get; set; }
	public EntityType Type { get; set; }
}

// simplified related data, A and B are equivalent here
public class Link
{
	public Entity Entity { get; set; }
	public Guid LinkedObjectId { get; set; }
	public DateTimeOffset LastAccessTimestamp { get; set; }
}

public interface ISomeRelatedDataRepository
{
	void AddLink(Link link);
	IEnumerable<Guid> GetLinkedObjectIds(Entity entity);
	void RefreshLastAccessTimestamp(Entity entity, Guid linkedObjectId);
}

Let’s look at another side - database layer. I want PostgreSQL to store A and B, related to A and B data separately

/* project "Db.Creator" */

CREATE TABLE entity_a (
	  id UUID PRIMARY KEY NOT NULL

	/* entity_a specific data */
	, prop_a1 VARCHAR NOT NULL
	, prop_a2 BOOLEAN NOT NULL
);
CREATE TABLE entity_b (
	  id UUID PRIMARY KEY NOT NULL

	/* entity_b specific data */
	, prop_b1 NUMERIC(4,2) NOT NULL
	, prop_b2 NUMERIC(4,2) NOT NULL
);

CREATE TABLE some_object (
	  id UUID PRIMARY KEY NOT NULL
	/* object properties */
);

/* same data structures for A and B */
CREATE TABLE some_data_entity_a (
	  id SERIAL PRIMARY KEY
	, entity_id UUID NOT NULL REFERENCES entity_a
	, linked_object_id UUID NOT NULL REFERENCES some_object
	, last_access_timestamp TIMESTAMP WITH TIME ZONE NOT NULL
);
CREATE TABLE some_data_entity_b (
	  id SERIAL PRIMARY KEY
	, entity_id UUID NOT NULL REFERENCES entity_b
	, linked_object_id UUID NOT NULL REFERENCES some_object
	, last_access_timestamp TIMESTAMP WITH TIME ZONE NOT NULL
);

because of

• consistency (foreign keys),
• performance (amount of data is going to be big for both A and B while usually I’ll want to read data only for A or only for B).

How ORM is expected to help

ORM layer looks like to be responsible for mapping general entity data to corresponding some_data_entity_X table. I exactly don’t want to write something like “if (entity.Type == Entity.TypeA) then … else if (entity.Type == Entity.TypeB) then …” many times or duplicate the same ORM code. I want:

• mapping to be done only once,
• implementations of the repositories to be clean from mapping stuff and don’t distinguish A and B when they are the same (I think it’ll be easier to read later).

Current solution

So I need to create a kind of abstract layer over real some_data_entity_X and save it from overwriting by t4 if possible. Currently I’m doing it in next way:

// project "Db.Linq2dbImplementation"

// TestLinq2db.generated.cs
// general data properties are removed from generated model
[Table(Schema="public", Name="some_data_entity_a")]
public partial class some_data_entity_a
{
	[associations go here]
}
[Table(Schema="public", Name="some_data_entity_b")]
public partial class some_data_entity_b
{
	[associations go here]
}

// TestContextManual.cs
// general data properties are in the base class, real class some_data_entity_X inherit it
public abstract class some_data_entity_base
{
	[PrimaryKey, Identity] public int id { get; set; } // integer
	[Column, NotNull] public Guid entity_id { get; set; } // uuid
	[Column, NotNull] public Guid linked_object_id { get; set; } // uuid
	[Column, NotNull] public DateTimeOffset last_access_timestamp { get; set; } // timestamp (6) with time zone
}
public partial class some_data_entity_a : some_data_entity_base {}
public partial class some_data_entity_b : some_data_entity_base {}

Mapping in verbose, but isolated and encapsulated way:

// project "Db.Linq2dbImplementation"

class Factory
{
	public static IMapper<some_data_entity_base> GetMapper(Entity entity) => GetMapper(entity.Type);
	public static IMapper<some_data_entity_base> GetMapper(EntityType type)
	{
		if (type == EntityType.TypeA)
			return new MapperA();
		if (type == EntityType.TypeB)
			return new MapperB();
		throw new ArgumentException($"No mapping for entityType='{type}'");
	}
}

interface IMapper<out TDb> where TDb : some_data_entity_base
{
	string TableName { get; }

	ITable<TDb> GetTable(TestContext db);
	TDb FromModelToDb(Link link);
}

abstract class MapperBase<TDb> : IMapper<TDb> where TDb : some_data_entity_base
{
	public abstract string TableName { get; }

	public abstract ITable<TDb> GetTable(TestContext db);
	public abstract TDb FromModelToDb(Link link);

	protected void FillDbFromModel(Link link, TDb toFill)
	{
		toFill.entity_id = link.Entity.Id;
		toFill.linked_object_id = link.LinkedObjectId;
		toFill.last_access_timestamp = link.LastAccessTimestamp;
	}
}

class MapperA : MapperBase<some_data_entity_a>
{
	public override string TableName { get; } = "some_data_entity_a";
	public override ITable<some_data_entity_a> GetTable(TestContext db) => db.some_data_entity_a;

	public override some_data_entity_a FromModelToDb(Link link)
	{
		var dbLink = new some_data_entity_a();
		FillDbFromModel(link, dbLink);
		return dbLink;
	}
}

class MapperB : MapperBase<some_data_entity_b>
{
	public override string TableName { get; } = "some_data_entity_b";
	public override ITable<some_data_entity_b> GetTable(TestContext db) => db.some_data_entity_b;

	public override some_data_entity_b FromModelToDb(Link link)
	{
		var dbLink = new some_data_entity_b();
		FillDbFromModel(link, dbLink);
		return dbLink;
	}
}

And the repository. It appears to be as clear as I want:

// project "Db.Linq2dbImplementation"

public class SomeRelatedDataRepository : ISomeRelatedDataRepository
{
	public void AddLink(Link link)
	{
		var mapper = Factory.GetMapper(link.Entity);
		var dbLink = mapper.FromModelToDb(link);

		using (var db = new TestContext())
			db.Insert(dbLink, mapper.TableName);
	}

	public IEnumerable<Guid> GetLinkedObjectIds(Entity entity)
	{
		using (var db = new TestContext())
		{
			var query = from link in Factory.GetMapper(entity).GetTable(db)
					where link.entity_id == entity.Id
					select link.linked_object_id;
			return query.ToList();
		}
	}

	public void RefreshLastAccessTimestamp(Entity entity, Guid linkedObjectId)
	{
		using (var db = new TestContext())
		{
			Factory.GetMapper(entity).GetTable(db)
			.Where(x => x.entity_id == entity.Id && x.linked_object_id == linkedObjectId)
			.Set(x => x.last_access_timestamp, DateTimeOffset.Now)
			.Update();
		}
	}
}

Questions

1. If I don’t specify “tableName” in Insert(…) extension

	public void AddLink(Link link)
	{
		var mapper = Factory.GetMapper(link.Entity);
		var dbLink = mapper.FromModelToDb(link);

		using (var db = new TestContext())
			db.Insert(dbLink/*, mapper.TableName*/);
	}

then insertion fails:

42P01: relation "some_data_entity_base" doesn't exist
   at Npgsql.NpgsqlConnector.<DoReadMessage>d__148.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.ValueTaskAwaiter`1.GetResult()
   at Npgsql.NpgsqlConnector.<ReadMessage>d__147.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at Npgsql.NpgsqlConnector.<ReadMessage>d__147.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.ValueTaskAwaiter`1.GetResult()
   at Npgsql.NpgsqlConnector.<ReadExpecting>d__154`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.ValueTaskAwaiter`1.GetResult()
   at Npgsql.NpgsqlDataReader.<NextResult>d__32.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Npgsql.NpgsqlDataReader.NextResult()
   at Npgsql.NpgsqlCommand.<Execute>d__71.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.ValueTaskAwaiter`1.GetResult()
   at Npgsql.NpgsqlCommand.<ExecuteNonQuery>d__84.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Npgsql.NpgsqlCommand.ExecuteNonQuery()
   at LinqToDB.Data.DataConnection.ExecuteNonQuery() in c:\projects\linq2db\Source\Data\DataConnection.cs:line 925
   at LinqToDB.Data.DataConnection.QueryRunner.ExecuteNonQueryImpl(DataConnection dataConnection, PreparedQuery preparedQuery) in c:\projects\linq2db\Source\Data\DataConnection.QueryRunner.cs:line 287
   at LinqToDB.Data.DataConnection.QueryRunner.ExecuteNonQuery() in c:\projects\linq2db\Source\Data\DataConnection.QueryRunner.cs:line 320
   at LinqToDB.Linq.QueryRunner.NonQueryQuery(Query query, IDataContextEx dataContext, Expression expr, Object[] parameters) in c:\projects\linq2db\Source\Linq\QueryRunner.cs:line 643
   at LinqToDB.Linq.QueryRunner.<>c__DisplayClass66.<SetNonQueryQuery>b__64(IDataContextEx db, Expression expr, Object[] ps) in c:\projects\linq2db\Source\Linq\QueryRunner.cs:line 633
   at LinqToDB.Linq.QueryRunner.Insert`1.Query(IDataContext dataContext, T obj, String tableName, String databaseName, String schemaName) in c:\projects\linq2db\Source\Linq\QueryRunner.Insert.cs:line 69
   at LinqToDB.DataExtensions.Insert[T](IDataContext dataContext, T obj, String tableName, String databaseName, String schemaName) in c:\projects\linq2db\Source\DataExtensions.cs:line 190
   at Db.Linq2dbImplementation.Repositories.SomeRelatedDataRepository.AddLink(Link link) in D:\GoogleDrive\C# Projects\TestLinq2db\Linq2dbImplementation\Repositories\SomeRelatedDataRepository.cs:line 24
   at Application.Program.Main(String[] args) in D:\GoogleDrive\C# Projects\TestLinq2db\Application\Program.cs:line 26

Looks like Insert(…) considers declared type (Factory.GetMapper(…) returns ITable<some_data_entity_base>), not real runtime type (actual type is always _a or _b). Is it a bug or feature 😃 ? There is similar closed issue #388. 2. In linq2db 1.0.7.3 Insert(…) with specified tableName didn’t insert to the correct table. First insert succeeded, but the second failed (if it was to another table, not to the same). Currently it works properly. Can I rely on this behaviour? 3. Do I use your library in the right way? Is there a better/simpler solution? Full code of my example is here: https://github.com/KasatkinaMariya/linq2dbAbstractMetaTable.

Environment details

linq2db version: 1.9.0
Database Server: E.g. PostgreSQL 9.5
Database Provider: E.g. Npgsql 3.2.5
Operating system: E.g. Windows 10
Framework version: .NET Framework 4.6.1

[sdanyliv]

Not for holy-wars

• make logic layer (contains declarations of main business logic entities, POCOs, interfaces (including repositories’ interfaces)) completely independent of ORM layer.

This one i really trade off. You can invent another way how to abstract from ORM, that already is abstraction over databases, but you will lose power and development speed.

• keep repositories’ role small and simple - creation of trivial queries (clever logic managers will build wanted “chains” from them later),

You will quickly find that building abstract “chains” will not help but complicates code and it’s support. Especially when performance is poor and you are trying to fix that.

• allow linq2db generate as efficient SQL as it is able,

You have to deal with IQueryable almost anywhere in your code that works with database. In that case linq2db will create optimized SQL for you. Using repositories kills that possibility, even you return IQueryable instead of IEnumerable

Check my post here, i’s just sample, how to deal with entities (in russian): http://rsdn.org/forum/dotnet/6879878.1 Almost whole thread is about using ORM Another thread about anti-pattern repository: http://rsdn.org/forum/dotnet/6877137

Currently when i start to develop solution, i’m trying to implement things quickly and simple: UI -> WebServer -> BL(with linq2db) During development you will find your own patterns and ways to reuse code and simplify new one for specific project. In other case you will spend time to create abstract framework, tuning fixing, throwing out unused code. For example you have created for TWO tables a tons of ineffective code 😉

My words for that again - try to write effectively from start, then find ways to simplify writing similar things.

Several facts:

• You will not change ORM, almost never - so why do you need abstraction from ORM?
• You almost never change Database Engine - linq2db is abstraction over Databases.
• BL Unit tests is better to run on real database. It’s slowdown testing BUT you are safe from Database Engine specific issues.

[KasatkinaMariya]

Yes, dictionary with O(1) is much more efficient and is more friendly to read. Now (after https://github.com/KasatkinaMariya/linq2dbAbstractMetaTable/commit/59b0ae4bdbfa1ad09c1eacadc60e1aecb140cb3e) generated Factory looks like this:

// TestLinq2db.generated.cs
...
public class Factory
{
	private static Dictionary<EntityType, Func<IMapper<DataEntityBase, LinkEntityBase>>> _enumValueToMapperFunc
		= new Dictionary<EntityType, Func<IMapper<DataEntityBase, LinkEntityBase>>>
		{
			{ EntityType.TypeA, () => new MapperTypeA() },
			{ EntityType.TypeB, () => new MapperTypeB() },
		};

	public static IMapper<DataEntityBase, LinkEntityBase> GetMapper(Entity entity) => GetMapper(entity.Type);
	public static IMapper<DataEntityBase, LinkEntityBase> GetMapper(EntityType type)
	{
		Func<IMapper<DataEntityBase, LinkEntityBase>> createMapper;
		if (_enumValueToMapperFunc.TryGetValue(type, out createMapper))
			return createMapper();

		throw new ArgumentException($"No mapping for entityEnumValue='{type}'");
	}
}
...

By the way, my bunch isn’t big enough to care about such optimizations at the moment. If it’s the biggest fail, then I’m happy to start using this construction to develop features valued by business.