Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.
And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.
ENH: show progress while fetching rows for query
See original GitHub issueAs reported by @QuinRiva in https://github.com/pydata/pandas-gbq/issues/12#issuecomment-391212920
Progress is written to logging while a query is running, but no progress is reported between when a query finishes and while the data is being downloaded to be added to a DataFrame. The problem is that we call list(rows_iter) to fetch all pages. Previously, progress was written as each page was downloaded.
A possible solution is to loop over rows_iter.pages instead. After the first page is fetched, the rows_iter.total_rows property is available, so it would be possible to display a percent complete or even use tqdm as done in https://github.com/pydata/pandas-gbq/pull/166.
Issue Analytics
- State:
- Created 5 years ago
- Comments:7 (4 by maintainers)
Top Results From Across the Web
Top Related Medium Post
Top Related StackOverflow Question
Troubleshoot Live Code
Top Related Reddit Thread
Top Related Hackernoon Post
Top Related Tweet
Top Related Dev.to Post
Top Related Hashnode Post
Now that
to_dataframeingoogle-cloud-bigquerysupportsprogress_bar_type, can we make use of that to add a progress bar toread_gbq? @tswastYeah, once I make a new release of
google-cloud-bigquery(I expect to next week), we can start populating the progress bar argument. I’d be okay always populating it as'tqdm', but it is nice to have the special version for notebooks, so an argument toread_gbqmakes sense.