Very high memory usage with write_html.py
See original GitHub issueOn my system running write_html.py
without arguments requires too much memory and takes too long. After more than 30 minutes I had to manually stop it because my system became unresponsive. Memory usage increased slowly but relentlessly, until write_html.py
used all 8 gigabytes of RAM plus 5 gigabytes of swap.
My data directory is currently 3.1 gigabytes. It will continue to expand in the future because I’m always fetching new subreddits.
How can I help debug this?
P.S. My knowledge of Python is still very small…
Issue Analytics
- State:
- Created 4 years ago
- Reactions:1
- Comments:10 (5 by maintainers)
Top Results From Across the Web
Python excessive memory usage with simple script
I am writing a very simple script that will count the number of occurence in a file. The file size is about 300Mb...
Read more >Very high memory usage on Windows 10 #12671 - GitHub
Very strange. I just tried a fresh install of python 3.8.5 with only scipy installed, and still got 1.5GB commit size. Can you...
Read more >Troubleshoot high memory use in an ElastiCache cluster
I'm seeing high or increasing memory usage in my Amazon ElastiCache cluster. How is memory usage determined on ElastiCache cluster nodes?
Read more >Optimizing Memory Usage in Python Applications
Find out why your Python apps are using too much memory and reduce their RAM usage with these simple tricks and efficient data...
Read more >listmonk - Free and open source self-hosted newsletter ...
High performance and features packed into one app. ... CPU usage is a fraction of a single core with peak RAM usage of...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
To give you a better idea, I have already archived 16 subreddits:
After commenting out those lines,
write_html.py
no longer uses too much memory. But I noticedindex.html
files for each subreddit are now missing, so I can’t display the archives with a browser.