Guidance on integrating linq2db with other code to manage connections

I am looking for any thoughts on how best to integrate / extend l2db, much like the linq2db.EntityFrameworkCore project.

Currently we have a home grown ORM that handles plain textual SQL, and I am looking to add Linq into it, ideally using l2db.

Our current system provides some features that we need to keep like caching, throttling, remote data over http, etc. It’s a huge system with thousands of queries, so I’m looking to support Linq alongside the current plain SQL. We are running against SQL Server.

I can probably hack something that works, but if possible wanted some guidance on how best to work with the l2db framework.

Our core challenges integrating l2db:

1. Generating the metadata without using T4 templates

2. Overriding the Command object that linq2db uses for retrieving data

   • Ideally I’d also provide the connection
   • I want to use the built in l2db mapping, so don’t just need the SQL text

1. Metadata & T4

The T4 issue is that we already have a migration system via scripts, that runs on CI, handles multiple replicating databases, etc, so T4 doesn’t work for our use case.

My current plan is to covert the T4 templates back into C# and host that logic inside our existing tool, unless there is an easier way you can think of ?

2. Overide connection / command

Investigations so far:

1. Override ExecuteReader in my LinqToDB.Data.DataConnection subclass to return a IDataReader from my own command subclass
   • The returned IDataReader is downcast internally e.g. if I create my own MiniProfiler command, I get Unable to cast object of type 'StackExchange.Profiling.Data.ProfiledDbDataReader' to type 'System.Data.SqlClient.SqlDataReader. This is occurring inside LinqToDB.Linq.IQueryRunner somewhere
   • This is also wasteful as l2db still has a connection, command, etc.
2. Looked at LinqExtensions.ProcessSourceQueryable, as it’s used here
   • I can see it uses reflection to extract the Db Context out of the EF.Core IQueryable
   • It then extracts the connection info from the Db Context, creating a DataConnection with the details
   • I can see the TransformExpression code, don’t really understand it, but assume all connections to the database are done by l2db itself rather than EF.Core.

Other ideas:

• Build my own DbConnection subclass like RetryingDbConnection.cs, but EnsureConnection is internal so I’m not sure how to plumb it in to the DataConnection.

  • Would you be open to making this protected or overridable in some way ?

• Build a IDataProvider that delegates to the built in SqlServerDataProvider, but overrides CreateConnection.

  • The overridded IDbConnection then implements custom code on CreateCommand

• Or in a similar approach to the IDataProvider, provide a IDbConnection factory func to the DataConnection

I’m guessing the Db Connection Func is the best approach, sound about right ?

---

Any help greatly appreciated and thanks for l2db, it’s great !

[sdanyliv]

My current plan is to covert the T4 templates back into C# and host that logic inside our existing tool, unless there is an easier way you can think of ?

As i know it is not a problem to run T4 generator from a code. But you can also generate model by your own code. A little bit mess but we did that for LINQPad - just extract needed code.

Override ExecuteReader in my LinqToDB.Data.DataConnection subclass to return a IDataReader from my own command subclass

We need sample project. In this case i can suggest solution or improve abstraction.

I can see the TransformExpression code, don’t really understand it, but assume all connections to the database are done by l2db itself rather than EF.Core.

Not exactly, we can reuse EF Core connection if it is compatible with our provider. TransformExpression just prepares Expression Tree to be compatible with our LINQ parser.

Other ideas:

Better to create PR with needed changes. For sure with tests - i can instruct how to setup Test project. We will review PR, propose changes and answer questions. You can do it only for SqlServer.

[davidglassborow]

Tried 2 more things, 1 which worked, 1 which didn’t.

1. Worked: Create my own DbConnection and related DbCommand / Parameters to track SQL and parameters. Uses a DataProvider subclass that prevent the dynamic casts in the expression evaluation to not cause problems:

    /* Simple provider that bypasses any issues with type checking */
    public class ProxyProvider : LinqToDB.DataProvider.SqlServer.SqlServerDataProvider
    {
        public ProxyProvider() : base("ProxyProvider", LinqToDB.DataProvider.SqlServer.SqlServerVersion.v2008) {}

        /* These two make sure no exceptions when checking overriden Db types */
        public override bool IsCompatibleConnection(IDbConnection connection) => true;
        public override Type DataReaderType => typeof(DbDataReader);
    }

1. Didn’t work: Implement by own IDataContext, with the expectation that by implementing

IQueryRunner IDataContext.GetQueryRunner(Query query, int queryNumber, Expression expression, object[] parameters)

I could manage the evaluation itself. However I ran into various issues around the public APIs not providing enough information for this to work. I’ll raise a separate issue for this.