question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Misleading log messages about opening connections

See original GitHub issue

Describe the bug

When dbt is parsing, it assigns a “connection name” for each model it’s parsing, just in case it’s needed. This goes through set_connection_name, which logs that it is Acquiring new postgres connection ... and Opening a new connection, .... This behavior was very reasonable before dbt lazy-loaded connections, because it’s the last centralized place before we end up in the adapter-specific open classmethod, so it’s a nice place to send that message.

Since dbt doesn’t generally need to access adapter connections during parsing, this log message is extremely misleading, and leads users who inspect the debug logs to believe that what’s taking a long time is opening a ton of connections. What’s actually (probably) taking a long time is parsing jinja. It would be good to know either way!

We should move the message about Opening a new connection into the part of LazyHandle that calls open.

I would like Acquiring to stay (though I’m very open to rewording it) as it’s a very useful message in the event of weird threading-related connection issues. We haven’t had any of those in a while, but I assume that as soon as we remove the message we’ll introduce a slew of connection management problems somehow…

Steps To Reproduce

  • install dbt
  • make a few simple models like select 1 as id
  • run dbt ls
  • watch dbt claim to acquire a bunch of connections

Expected behavior

At the end of the day, I don’t expect dbt to log that it’s “Opening” a connection unless we’re opening a socket (or at least trying to).

Screenshots and log output

$ dbt ls
...
2020-06-03 16:46:16.243448 (MainThread): Opening a new connection, currently in state init
2020-06-03 16:46:16.253313 (MainThread): Acquiring new snowflake connection "test.project.some_test".

System information

Any database, any supported python version, any OS, dbt >= 0.15.1

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Reactions:2
  • Comments:5 (5 by maintainers)

github_iconTop GitHub Comments

1reaction
southpolemonkeycommented, Jun 4, 2020

I’d like to take this issue if it hasn’t been assigned yet.

1reaction
beckjakecommented, Jun 3, 2020

@jtalmi You asked if we had any more good first issues - I think this is a great one! I know that was a month ago, but let me know if you’re interested in picking this up.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Misleading log messages about opening connections #2511
Describe the bug When dbt is parsing, it assigns a "connection name" for each model it's parsing, just in case it's needed.
Read more >
Fix connection errors - Google Chrome Help
Get help with a specific error message · Step 1: Sign in to the portal · Step 2: Open the page in Incognito...
Read more >
Protect yourself from phishing - Microsoft Support
Cybercriminals have been successful using emails, text messages, ... that an email message is a scam, don't open any links or attachments that...
Read more >
Troubleshoot - FAQs - Kemp Technologies
Log Message : "Connection Failed - unreachable host". There is no route to the host, machine is probably down, or a network failure....
Read more >
Log messages - IBM
This may be caused by attempting to connect to the wrong URL, by a network error, or by some kind of CRL server...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found