Memory use is increasing as the times of call in function scipy.stats._sobol._categorize

My issue is about memory leak in function scipy.stats._sobol._categorize. As the below code shows, if I input illegal input to the function, the memory use is increasing as the number of calls.

Reproducing code example:

import sys
import tracemalloc
import numpy
import scipy.stats._sobol

arg_list = []
arg_list.append(-87.11965751389266)
arg_list.append(17.04812581438513)
arg_list.append(-3)
arg_list.append(-87)
arg_list.append(-14)
arg_list.append(37)
arg_list.append(-63)
arg_list.append(-73)
arg_list.append(-83)
arg_list.append(-14)
arg_list.append(96)
arg_list.append(70)

memory_recorder = []
tracemalloc.start()
start_snapshot = tracemalloc.take_snapshot()
for i in range(0, 100):
  try:
    scipy.stats._sobol._categorize(numpy.array([arg_list[0], arg_list[1]]),
                                   numpy.array([arg_list[2], arg_list[3], arg_list[4], arg_list[5], arg_list[6]]),
                                   numpy.array([arg_list[7], arg_list[8], arg_list[9], arg_list[10], arg_list[11]]))
  except:
    pass
  if i % 10 == 0:
    one_snapshot = tracemalloc.take_snapshot()
    memory_recorder.append(one_snapshot.compare_to(start_snapshot, 'lineno')[0])

for i in range(2, len(memory_recorder)):
  if str(memory_recorder[i].traceback) == str(memory_recorder[1].traceback):
    print(memory_recorder[i].traceback)
    print(memory_recorder[i].size)

tracemalloc.stop()

Error message:

test.py:32
9594
test.py:32
13690
test.py:32
17482
test.py:32
21482
test.py:32
25274
test.py:32
29322
test.py:32
33114
test.py:32
37210

Scipy/Numpy/Python version information:

scipy 1.7.0

[peterbell10]

I’ve reported the leak to cython (cython/cython#4296) but agree this particular function shouldn’t cause any issues. The issue @tupui points out with base_draws doesn’t matter because it’s the first argument so no other memory views will have been constructed yet.

[tupui]

Thanks for having a deeper look @peterbell10