question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Support parallel Physical Connection creation for multi-threaded scenarios

See original GitHub issue

I’m trying to spin up 50 threads to immediately open 50 SqlConnections to a Microsoft SQL Server connection pool. However, no matter what I try, the SqlConnections seem to only open one at a time. In other words, if each connection takes 2 seconds to open, then the 50’th thread’s Open method will take 100 seconds to complete. Further to this point, if the Connection Timeout isn’t set to > 100 seconds then the connection attempts will eventually start timing out. With the default Connection Timeout = 30, the 16’th thread’s Open times out.

To put this another way, it seems like the 1st SqlConnection.Open takes 2 seconds, the 2nd SqlConnection.Open takes 4 seconds, the 3rd SqlConnection.Open takes 6 seconds… the 16th SqlConnection.Open takes 32 seconds and causes a timeout exception.

I also tried setting Min Pool Size=50 and just opening a single SqlConnection, and that has the same behavior. There seems to be a background process that starts opening connections until it gets to 50, but the 16th one times out because it’s over the 30 second connection timeout.

It’s like there’s something blocking internally to only allow this one-at-a-time addition to the connection pool. I’ve asked around on stack overflow, and the responses seem to suggest that if you open 50 connections in parallel, they should all open.

So this makes me wonder if this is a .Net Core bug or “feature”.

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Comments:33 (17 by maintainers)

github_iconTop GitHub Comments

2reactions
saurabh500commented, Jun 12, 2020

@cheenamalhotra @OQO FWIW this serial opening of concurrent connections for the same pool, even in case of multithreaded scenario is intentional and by design of the Connection Pool.

From the past owners of SqlClient I understand that the idea was to not overwhelm a SQL server because of a flood of connections from the single Connection Pool which needs many connections as well as many clients trying to connect to a single SQL server when failover is in progress. The recommendation is to use MinPoolSize in Connection String for connection pool and allow the pool to warm up to the MinPoolSize after the first connection attempt is made. This is something that can be tried. After that the connection pool will try to maintain MinPoolSize count of connections in the pool.

As of whether the behavior should be changed or not, it may be OK to relax the requirement for Azure SQL DBs, but it may be good to keep this behavior for on-prem SQL servers. There are scenarios where a server fails over and multiple clients with connection pools are trying to login to the same server. In those cases, we may not want the server overwhelmed by a slew of connections coming in.

0reactions
OQOcommented, Jun 13, 2020

@cheenamalhotra Thank you. After I asked, I guessed the same so I already did. Please see #601

Read more comments on GitHub >

github_iconTop Results From Across the Web

Is it possible to run a table to table mapping scenario in ...
Is it possible to run a table to table mapping scenario in parallel (multi threading) we have a huge table and we already...
Read more >
217-29: Threads Unraveled: A Parallel Processing Primer
Reviewing the underpinnings of threading will help you understand what parallel processing is and give you insights into the performance limits of threaded ......
Read more >
Chromium Docs - Threading and Tasks in Chrome
Threads ; Tasks; Prefer Sequences to Physical Threads. Posting a Parallel Task. Direct Posting to the Thread Pool; Posting via a TaskRunner.
Read more >
Executing multiple subJobs in parallel - Cloud - 8.0
The Multi thread execution feature allows you to run multiple subJobs that are active in the workspace in parallel.
Read more >
Why using more threads makes it slower than using less ...
As a rule of thumb, the more of you program that you cannot transform in parallel execution, the less you can obtain using...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found