`Session is locked by a concurrent client` from bulk copy

I’m not sure if it is upstream issue in ClickHouse or some races in provider code, so I will fill it here for now.

Following code fails with Session is locked by a concurrent client error for me after ~200 iterations, which is quite annoying as it fails test run at random locations…

CH version: 22.6.3.35 Sessions: enabled

            using var cn = new ClickHouseConnection(cs);
            cn.Open();
            using var cmd = cn.CreateCommand();
            cmd.CommandText = "drop table if exists test_table";
            cmd.ExecuteNonQuery();
            cmd.CommandText = "create table test_table(field Int32) Engine=Memory";
            cmd.ExecuteNonQuery();

            var cnt = 0;
            var values = new object[1][] { new object[1] };
            while (true)
            {
                using var bc = new ClickHouseBulkCopy(cn);
                bc.MaxDegreeOfParallelism = 1;
                bc.DestinationTableName = "test_table";
                values[0][0] = cnt;
                await bc.WriteToServerAsync(values);
                cnt++;
            }

[DarkWanderer]

Just to give an update. I’ve figured out the cause for the error - it is because ClickHouse doesn’t like it when calls from same session come from different HTTP connections, even if the queries never come concurrently. Workaround is “simple” - just pass a HttpClient with a single-connection HttpClientHandler:

            new HttpClientHandler()
            {
                AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate,
                MaxConnectionsPerServer = 1,
            };

Fixing that in scope of the library is a more complex story, however - I’m thinking how to do that in a fashion which is both reasonably backward-compatible and restorative towards this issue

[joshbartley]

Recently I looked into the HttpClient and Handler docs and recommendations. The one slightly mentioned, that the Azure client uses, is the SocketsHttpHandler which HttpConnectionHandler uses internally. You wouldn’t rely on the consumer to pass in a HttpClient and/or HttpConnectionHandler as you would have an internal one that handles sockets for you based on the endpoint name. You would need a list of all the connection to pool them, and the SocketsHttpHandler can drop a connection after X time as well to handle dns changes. .net 5 has more exensibility for the Sockets handler as well when 3.1 doesn’t but similar functionality. I’ll look around in the connection code though I don’t know of an easy way to test any style of those changes without a complicated container setup. The Sql Connection pool classes have a lot of history in them that makes me think there are some edge cases that only time will show in any connection pool class.