question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

memory usage after running scipy.signal.fftconvolve (possible memory leak?)

See original GitHub issue

Consider the memory usage of the following code after executing each statement:

import numpy
import scipy.signal
import gc

# memory at this point is ~35mb

a = numpy.ones(10**7)
b = numpy.ones(10**7)

# memory at this point is ~187mb

c = scipy.signal.fftconvolve(a, numpy.flipud(b), mode="full")

# memory usage at this point is 645mb
# given that a,b,c take up about 305mb in total, this is much
# larger than expected

# If we delete a,b,c and garbage collect...

del a,b,c
gc.collect()

# ...the memory usage drops to ~340mb which is a drop of 305mb
# (as expected since that is the size of what we deleted)
# but the remaining memory usage is still 340mb
# which is much larger than the starting value of 35 mb

Is this extra several hundred mb of memory usage a bug?

Platform:

  • CPython 3.5.1 x64
  • Scipy 0.17.0 (Christoph Golke’s package)
  • Numpy+MKL 1.11.0rc1 (Cristoph Golke’s package)
  • Windows 8.1 x64

Issue Analytics

  • State:open
  • Created 8 years ago
  • Comments:23 (17 by maintainers)

github_iconTop GitHub Comments

1reaction
souravsinghcommented, Jan 14, 2019

@rgommers I have confirmed that the cache functions work as required. The docs may need correction on the Cache part and we will need to make a function for the cache destruction.

0reactions
mreineckcommented, Aug 17, 2019

Good point!

Read more comments on GitHub >

github_iconTop Results From Across the Web

Memory usage in Numpy - python - Stack Overflow
I'd try the following things in order: Use regular convolve instead of fftconvolve. Use scipy.weave and do your own convolve operation (Thus ...
Read more >
SciPy 1.4.0 Release Notes — SciPy v1.9.3 Manual
has linear memory complexity, improved performance, and supports ... Like scipy.signal.fftconvolve , this function supports specifying dimensions along ...
Read more >
scipy.signal.fftconvolve — SciPy v1.9.3 Manual
Convolve two N-dimensional arrays using FFT. Convolve in1 and in2 using the fast Fourier transform method, with the output size determined by the...
Read more >
SciPy 1.0.0 Release Notes — SciPy v1.9.3 Manual
The idea of SciPy was already in multiple people's minds in 1997 when I first ... code does not use deprecated SciPy functionality...
Read more >
SciPy Reference Guide - Numpy and Scipy Documentation
#7214: Memory use in integrate.quad in scipy-0.19.0 ... The function scipy.signal.correlate and scipy.signal.convolve have a new optional ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found