question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Do we need to SELECT 1 before each query?

See original GitHub issue

With the recent clustering release, we’re looking at whether we can use BQ for queries which previously required an indexed DB.

For these, the overhead is more important than the throughput. The overhead is high - in the case that the query is cached (if not, add 1s to all these, for a 700 row query on a small cluster):

  • 73ms: query time reported by BQ
  • 1160ms: query time logged by pandas-gbq
  • 3100ms: wall time reported by %timeit -n 1 -r 1 gbq.read_gbq(query)

The latter two are particularly far apart. One of the reasons for this is that we issue a SELECT 1 query before each query the user submits. This happens as part of _try_credentials: https://github.com/pydata/pandas-gbq/blob/e753cc4e61b0089dcea6330eb8c77e93a62984a7/pandas_gbq/auth.py#L281

Is this required? Could we instead attempt the query, and only attempt different auth methods if there’s a failure?

Issue Analytics

  • State:closed
  • Created 5 years ago
  • Comments:5 (2 by maintainers)

github_iconTop GitHub Comments

2reactions
tswastcommented, Aug 21, 2018

Also in #161, I’d like to cache the credentials after the first call in a global Context object to avoid subsequent unnecessary auth calls.

1reaction
max-sixtycommented, Sep 5, 2018

I think we can close this - now it’s a single call per session?

Read more comments on GitHub >

github_iconTop Results From Across the Web

What does it mean by select 1 from table? - Stack Overflow
select 1 from table will return the constant 1 for every row of the table. It's useful when you want to cheaply determine...
Read more >
Why is the Select before the From in an SQL query? [closed]
In the case of select, you determine which attributes you want, then add the tables ... it can usually be done at the...
Read more >
The Purpose of WHERE 1=1 in SQL Statements - Navicat
The condition obviously means WHERE TRUE, so it's just returning the same query result as it would without the WHERE clause.
Read more >
Learn SQL: SELECT statement
The SELECT keyword is the only one required in the SELECT statement ; You can perform mathematical operations in SQL (SELECT 1+2;). If...
Read more >
Create a simple select query - Microsoft Support
When you want to select specific data from one or more sources, you can use a select query. A select query helps you...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found