question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Reduce refresh_queries's memory usage

See original GitHub issue

refresh_queries looks up all outdated queries and then schedules refresh tasks for each. The lookup can take a while and consume a lot of memory. We could probably do this in batches or yield for each outdated query.

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Comments:5 (5 by maintainers)

github_iconTop GitHub Comments

2reactions
arikfrcommented, Oct 28, 2019

Seems like it might be as simple as calling yield_per on the SQLA query object.

0reactions
arikfrcommented, Nov 3, 2019

👍

Read more comments on GitHub >

github_iconTop Results From Across the Web

Excel - Power queries eating up all RAM and taking ages to load
The file takes ages to load (I set up all data connections with Refresh data when opening the file checked), with 10-11 GBs...
Read more >
Power Query for Excel: Load Settings Minimize Wasted Memory
To change the Default Query Load Settings, choose Options from the Machine Settings section of the Power Query ribbon or in the File...
Read more >
Very high memory consumption on refresh
HI, I have very high memory consumption when I load from Power Query to PBI Desktop or when I'm doing plain refresh. My...
Read more >
Performance Tip for Power BI; Enable Load Sucks Memory Up
The best approach is to disable loading before closing query editor. Disabling Load doesn't mean the query won't be refreshed, it only means...
Read more >
Tuning for Refresh - Vertica
Set MEMORYSIZE in the REFRESH pool to a fixed value. The Resource Manager then tunes the refresh query to only use this amount...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found