dzsave of a large array of images

I have a large array (maybe thousands) of images, that I’m joining first:

merged = pyvips.Image.arrayjoin(images, across=column_count)

and then trying to dzsave:

merged.dzsave(
    os.path.join(tmpdir, "deepzoom"),
    overlap=0,
    depth="onetile",
)

It seems, that dzsave requires all images to be simultaneously opened, because I’m getting

vips__file_open_read: unable to open file "/tmp/tmpn_kyd4w4/21/23.jpg" for reading
unix error: Too many open files

There is a 1024 opened files limit on my system which I don’t want to change.

The problem here is that even though any single tile does require only a small subset of opened files, libvips still opens all of them. It seems that this might be optimized.

Does it make sense? Is it hard to support this optimization?

[jcupitt]

Yes, you can use pyvips.Image.black(100, 100) to make a black image to add to your arrayjoin inputs. You can reuse the same image many times, eg.:

empty = pyvips.Image.black(100, 100)
mosaic = pyvips.Image.arrayjoin([t1, t2, empty, t3, t4, empty, empty, t5], across=2)
mosaic.dzsave(...)

It should be quick. The black images never really exist, they just supply pixels on demand as required by dzsave.

[al-muammar]

That’s great, thanks for suggestions! That should be definitely enough.