Use the same base sharp instance for multiple transformation chains

I’m currently working on a server that delivers images manipulated by sharp under the hood.

We expect that several different chains of transformations will be requested in short windows of time for the same base image (pretty much concurrently when you take into account the time to read said base image) but, afaik, each transformation chain requires its own sharp instance.

The first obvious drawback is that libvips will use as many times the resources for the exact same image (it will decode and store the same base image for each transformation chain).

The less obvious drawback is that it makes it impossible to use a ReadableStream and pipe it into the sharp instance. We have to flush the image into a Buffer so that we can create other instances when needs be without reading the base image every time, meaning more memory consumption (only, this time, within V8) and more latency for any transformation chain requested while the image is being read.

Would it be possible to add some means of cloning sharp instances and avoid those caveats ?

In order not to break everything, how about factory instances, separate from current sharp instances?

var factory = sharp.factory();

readableStream.pipe( factory );

factory.newInstance().resize( 800, 600 ).pipe( firstWritableStream );
factory.newInstance().extract( 20, 20, 100, 100 ).pipe( secondWritableStream );

Do you think this would be feasible ?

[lovell]

Commit 01b4d6e on the knife-clone branch (itself a branch of knife) provides the experimental clone method that allows the following slight modification of Julian’s original example to work:

var factory = sharp();

readableStream.pipe( factory );

factory.clone().resize( 800, 600 ).pipe( firstWritableStream );
factory.clone().extract( 20, 20, 100, 100 ).pipe( secondWritableStream );

Cloned instances share input data with their siblings so this approach is fairly memory efficient.

This can be tested via npm install lovell/sharp#knife-clone.

The functional tests contain an example where two cloned instances inherit the same angle of rotation but differ in resize dimensions.

[jcupitt]

Hi all, regarding @LinusU 's note on multiple output images:

1. Specify input stream / file
2. Specify list of “transform pipelines”
3. Run all of the pipelines at the same time 1. Let libvips read the first chunk into memory 2. Use the first chunk as input in every pipeline 3. Repeat with next chunk until done

I can see that this might be possible. libvips currently has a thing called vips_sequential() which does thread reordering: it tracks the Y position as the set of threads in a pipeline scan down an image and stalls threads which get too far ahead. Combined with a small cache, you can make a set of random threads request pixels from the input in a strictly sequential way. This is how libvips is able to stream from libraries like libjpeg:

https://github.com/jcupitt/libvips/blob/master/libvips/conversion/sequential.c

I think you’d need to extend vips_sequential() to count the number of times each scanline should be served up. The rule would be something like: stall requests for new scanlines until the oldest line in cache has been requested N times, where N is the number of simultaneous output images. It would need some experimentation!

@jaubourg mentioned streaming. libvips will stream most formats to and from files (though not webp, sadly, due to libwebp problems), but needs to have the whole image there at once for memory sources and sinks. There’s a libvips branch which adds true streaming for memory sources and sinks too, see #30 and #179, if you’ve not come across it.