RGB/rasterize not releasing memory properly

ALL software version info

holoviews ‘1.13.4.post4+g8f7c67f70’ (current master)

Description of expected behavior and the observed behavior

I have an an image that I’m using hv.RGB and rasterize to render. Everytime I render the image, my memory usage for the python kernel increases incrementally. This incremental change matches with what tracemalloc tells me the image construction is consuming. Essentially, the memory allocated for the image is never released. I have to restart my kernel to release the memory.

Complete, minimal, self-contained example code that reproduces the issue

A test image can be downloaded from unsplash, for example this one. Be sure to get the original size. Or you can immediately download the file if you trust me enough to start the download directly 😃

This bit of code will tell us that we need about 48 MB of memory to run this

from PIL import Image
import tracemalloc
import holoviews as hv
hv.extension('bokeh')

path = 'shifaaz-shamoon-ikVBaapsTPg-unsplash.jpg'

tracemalloc.start()
with open(path, 'rb') as f:
    image = np.array(Image.open(f))
img = hv.RGB(image)    
rasterize(img)
snapshot = tracemalloc.take_snapshot()
top_stats = snapshot.statistics('lineno')
print("[ Top 10 ]")
for stat in top_stats[:10]:
    print(stat)

Now, if you repeatedly call the following code:

with open(path, 'rb') as f:
    image = np.array(Image.open(f))
img = hv.RGB(image)    
rasterize(img)

You’ll see that every time it’s run (I’m using jupyter to visualize), the memory consumption on your machine increases by 48 MB.

If you remove the rasterize(img) step, you won’t see any memory increase. I suppose this would be expected since it isn’t rendered, the object isn’t populated in the same way?

Also, if only call the PIL portion like this:

with open(path, 'rb') as f:
    image = np.array(Image.open(f))

You can see that this bit is properly releasing memory. So it seems to be and issue with the hv.RGB object.

[philippjfr]

I no longer believe this to be a real issue, we have been running an application which uses rasterize/datashade for over a month now without leaking memory.

[jlstevens]

I’ll assign this to the next milestone (from v1.13.x) - if it turns out to be a difficult, fundamental problem to solve we can bump it back