question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

file stays in memory / doesn't close?

See original GitHub issue

I’m not sure what is going on exactly, but I suspect that files are being kept in memory. When reading a large number of files (~1000 files) in a loop for example:

i = 0
for file in file_list:
    file2read = mypath + '//' + file
    las = lasio.read(file2read)
    lasDF=las.df()
    myIndex = lasDF.index
    lasDF['DEPTH'] = myIndex
    lasDF['LIN_RES'] = np.log(lasDF['RESD_FINAL'])
    output = np.concatenate((output, lasDF), axis=0)
    i+=1
    print(i/len(file_list)*100, "% complete. Read: ", file)

The memory usage seems to continuously climb. Is there a file.close() that needs to be performed by the user?

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:7 (4 by maintainers)

github_iconTop GitHub Comments

1reaction
danomics-camcommented, Nov 16, 2020

Version is: ‘0.24.1’ Some add’l info if it makes any difference: OS: Windows 10; 16GB Ram Total LAS file count: 1464 Total LAS input file size: 1.21 GB I’m wondering if it’s not getting stuck in the python garbage collector. For example I just ran about 455 files through and checked the gc.get_count() and it’s at (557,6,1), so might not be a lasio issue, but a setting on my side.

0reactions
kinverarity1commented, Nov 16, 2020

No worries, glad you have found the issue! 🚀

Read more comments on GitHub >

github_iconTop Results From Across the Web

File stays in memory after being closed - python - Stack Overflow
The issue seems to be in the read() function as a pass does not cause any memory usage growth. While looping through the...
Read more >
Windows 10 not releasing standby memory when required.
My system has 16GB of RAM and a max pagefile of 1000MB. The standby memory cache stays around 7GB even when my commit...
Read more >
Memory held by the resources is not released even after ...
Hi,. I created a WPF application with byte array, memory stream and BitmapImage when a main page is loaded as in the below...
Read more >
Windows 11 has a memory leak bug and here's how to fix it
If not, go ahead and close all the File Explorer instances you've opened (just right click the folder icon in the taskbar and...
Read more >
Matlab doesn't release memory when variables are cleared
you can write a script to close matlab and continue with the next function in a new file, after saving variables to disk....
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found