Node.js - abort(RangeError: WebAssembly.instantiate(): Out of memory: wasm memory)

Hello 👋

I met this error when I try to process serveral image in parallel :

ERROR [15:10:28.879]: abort(RangeError: WebAssembly.instantiate(): Out of memory: wasm memory). Build with -s ASSERTIONS=1 for more info.
    RuntimeError: abort(RangeError: WebAssembly.instantiate(): Out of memory: wasm memory). Build with -s ASSERTIONS=1 for more info.
        at abort (/app/node_modules/@squoosh/lib/build/index.js:24:9428)
        at /app/node_modules/@squoosh/lib/build/index.js:24:11293
failed to asynchronously prepare wasm: RangeError: WebAssembly.instantiate(): Out of memory: wasm memory

I used this code

const imagePool = new ImagePool();

const image = imagePool.ingestImage(`./public/${req.params.s3Name}`);
await image.decoded;

const preprocessOptions = {
  resize: {
    // method:'lanczos3',
    enabled: true,
    height: 300,
  },
};
// eslint-disable-next-line no-console
console.time(`preprocess ./public/${req.params.s3Name}`);
await image.preprocess(preprocessOptions);
// eslint-disable-next-line no-console
console.timeEnd(`preprocess ./public/${req.params.s3Name}`);

const encodeOptions = {
  mozjpeg: {
    quality: 50,
  }, //an empty object means 'use default settings'
};

// eslint-disable-next-line no-console
console.time(`encode ./public/${req.params.s3Name}`);
await image.encode(encodeOptions);
// eslint-disable-next-line no-console
console.timeEnd(`encode ./public/${req.params.s3Name}`);
imagePool.close();

const extension = req.params.s3Name.split('.').pop();
res.set('Content-Type', `image/${extension}`);

const rawEncodedImage = (await image.encodedWith.mozjpeg).binary;

fs.writeFileSync(`./public-light/${req.params.s3Name}`, rawEncodedImage);

Several information about this issue :

• although this issue is thrown, once I have catch it, the image seems to be processed anyway.
• although this code was under a classic try/catch, the error crashed Node.js process. I had to use this code to really catch the error :

process.on('uncaughtException', (err) => {
  logger.error(err);
});

Then, with this code, the process does not crash anymore and seems to continue its operation.

I have on route that ask a picture, reduce it (with the previous code) and return them. The erreur appears when I have multiple call at the same time.

I hope these information can be helpful for you. Tell me if you need another context data.

Thank you for your work 😃

[jcao219]

I’m able to reproduce the same error message with the following minimal code:

const { ImagePool } = require('@squoosh/lib');
async function processImage() {
    const imagePool = new ImagePool();
    await imagePool.ingestImage('kitty.jpg').decoded;
    imagePool.close();
}
for (let i = 0; i < 10; i++) {
    processImage();
}

However, the problem doesn’t occur if I reuse the ImagePool:

const { ImagePool } = require('@squoosh/lib');
const imagePool = new ImagePool();
async function processImage() {
    await imagePool.ingestImage('kitty.jpg').decoded;
}
const promises = [];
for (let i = 0; i < 10; i++) {
    promises.push(processImage());
}
Promise.all(promises).finally(() => imagePool.close());

Alternatively, if reuse of the ImagePool is not possible, I can fix the issue by passing in a limited amount of workers:

const { ImagePool } = require('@squoosh/lib');
async function processImage() {
    const numWorkers = 1;
    const imagePool = new ImagePool(numWorkers);
    await imagePool.ingestImage('kitty.jpg').decoded;
    imagePool.close();
}
for (let i = 0; i < 10; i++) {
    processImage();
}

Do either of those two workarounds might apply to you, @Nick-Mazuk and @Tchoupinax?

Meanwhile, I can dig deeper into why multiple ImagePools can easily run out of memory.

[surma]

The ImagePool manages the worker, and the worker won’t know when it is not getting any more messages. So yes, closing the pool is necessary for a clean teardown.