question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Superset memory leak

See original GitHub issue

A clear and concise description of what the bug is. After querying the same chart for a long time, the available memory continues to decrease until the system runs out of memory

Expected results

Recycle memory after querying the chart

Actual results

The available memory keeps decreasing

Screenshots

If applicable, add screenshots to help explain your problem. I started a scheduler job to query chart image

How to reproduce the bug

1.force refresh of the chart in Dashboard 2. use memory profile image 3. see log

image After many times image

use gc.collect() at end line, But the memory is not reclaimed

Environment

  • superset version: 1.1.0
  • python version: 3.7.9
  • node.js version: 12.18.3

Issue Analytics

  • State:open
  • Created 2 years ago
  • Comments:11 (7 by maintainers)

github_iconTop GitHub Comments

1reaction
villebrocommented, Oct 20, 2021

Thanks for investigating this @chen-ABC ! I can try to repro and help get this fixed and released soon if this is indeed the culprit!

1reaction
chen-ABCcommented, Oct 20, 2021

@zhaoyongjie
I think I found the cause of the memory leak.

 @utils.memoized(watch=("impersonate_user", "sqlalchemy_uri_decrypted", "extra"))
    def get_sqla_engine(

memoized use tuple key , The keys for the same query are different. Because there’s an object in the key image

I think we have the same problem in superset 1.3.0 example: image

Read more comments on GitHub >

github_iconTop Results From Across the Web

airbnb/superset - Gitter
@AnithaMurugesan I was just curious about what hardware specs you are running Superset on, things like RAM, CPU cores, etc. Things will be...
Read more >
Preset Cloud — The Visualization Layer for the Modern Data ...
Preset provides a managed service of Apache Superset, an open source business intelligence software which Maxime Beauchemin started in at Airbnb ...
Read more >
MemLab: An open source framework for finding JavaScript ...
We've open-sourced MemLab, a JavaScript memory testing framework that automates memory leak detection. Finding and addressing the root cause ...
Read more >
How To Detect and Prevent Memory Leaks | Scout APM Blog
The above example is likely to cause a memory leak because the variable requests, which holds a new instance of the Map object,...
Read more >
[GitHub] [incubator-superset] yamyamyuo commented on issue ...
[GitHub] [incubator-superset] yamyamyuo commented on issue #5930: superset memory leak? ... @vnnw thanks! it works! [ Full content available at: https://github.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found