Repeatedly calling scipy.signal.resample uses more memory than expected

I have code that resamples many audio files using scipy.signal.resample. The code uses much more memory than I would expect to the point of freezing the OS. I’ve managed to simplify my code to a simple test case. This uses ~8Gb RAM after about 6 iterations but I would expect it should use much less memory than this.

I am using inputs of varying sizes, which seems to be significant. If I use an audio input of constant size there doesn’t seem to be a problem.

from scipy.signal import resample
import numpy as np

def Test():

	shps = range(11473744, 1000, -2)

	for i, shp in enumerate(shps):
		signal = np.random.normal(size=(shp, 1))

		print (i, signal.shape)
		targetSamples = signal.shape[0]*44100//48000
		signal2 = resample(signal, targetSamples, axis=0)
		print (signal2.shape)
		del signal, signal2

if __name__=="__main__":
	Test()

I might have missed something obvious, so this might not even be a bug!

• Scipy 1.4.1
• Numpy 1.18.4
• Python sys.version_info(major=3, minor=6, micro=9, releaselevel=‘final’, serial=0)

[mreineck]

I’m not absolutely sure, but if the resampling process somehow calls scipy.fft internally, it might be caused by the module’s internal plan caching. Currently scipy.fft caches plans for the last 16 different transforms it did, and for 1D transforms they are comparable in size to the transformed arrays (for “difficult” transform sizes even larger).

One way to test this hypothesis is to start your test with a smaller size and check whether the memory consumption still increases after 16 iterations.

[peterbell10]

Indeed, it is about 1/3 speedup now that I test it again. In that case, caching the factors seperately is probably not a good idea since it means the bluestein plans won’t be evicted from cache by regular sized plans, so will stick around for longer.