gatsby-plugin-sharp: WorkerError @2.6.27

Description

Error processing a .png file when running gatsby develop

Steps to reproduce

Upgrading gatsby-plugin-sharp to version 2.6.27

Expected result

Images would be queried as expected

Actual result

Environment

System: OS: Linux 5.4 Ubuntu 20.04.1 LTS (Focal Fossa) CPU: (8) x64 AMD Ryzen 7 2700U with Radeon Vega Mobile Gfx Shell: 5.0.17 - /bin/bash Binaries: Node: 12.16.1 - ~/.nvm/versions/node/v12.16.1/bin/node Yarn: 1.22.4 - /usr/bin/yarn npm: 6.13.4 - ~/.nvm/versions/node/v12.16.1/bin/npm Languages: Python: 2.7.18 - /usr/bin/python Browsers: Chrome: 84.0.4147.105 Firefox: 80.0 npmPackages: gatsby: ^2.23.12 => 2.24.53 gatsby-background-image: ^1.1.1 => 1.1.2 gatsby-image: ^2.4.15 => 2.4.16 gatsby-plugin-google-analytics: ^2.3.13 => 2.3.13 gatsby-plugin-layout: ^1.3.10 => 1.3.10 gatsby-plugin-manifest: ^2.4.21 => 2.4.27 gatsby-plugin-module-resolver: ^1.0.3 => 1.0.3 gatsby-plugin-offline: ^3.2.21 => 3.2.26 gatsby-plugin-react-helmet: ^3.3.10 => 3.3.10 gatsby-plugin-react-svg: ^3.0.0 => 3.0.0 gatsby-plugin-sharp: 2.6.26 => 2.6.26 gatsby-source-filesystem: ^2.3.24 => 2.3.27 gatsby-source-graphql: ^2.7.1 => 2.7.2 gatsby-transformer-sharp: ^2.5.13 => 2.5.14

[diegoolavarria]

My current solution is to downgrade gatsby-plugin-sharp to version 2.6.26.

[davet42]

NOTE: this was a preliminary post as I was chasing the problem with my system. Eventually I realized by instrumenting and running the pngquant binary manually that everything was working. It turns out that the imagemin-pngquant has special handling to look for exit code “99”. Unfortunately, the way to access this change from error.code to error.exitCode.

When I forced the verbose option and added some logging in the (mentioned below) index.js routine, I realized that “error.code” was returning EPIPE (means connection no longer there) or undefined. Since it always worked (regardless of quality parameter) when I ran pngquant manually from command line, I assumed there was a problem with the invocation of the binary through execa.

BTW the package in question is at https://github.com/imagemin/imagemin-pngquant The underlying binary is at https://pngquant.org/ In the notes on binary, you will see the standard (for this package) practice of returning exit code 99 whenever the output ends up being the same as the input due to characteristics of the input image and the requested quality parameter. The imagemin-pngquant routine was trying to catch this. But, we need to make a change to the reference as shown below.

Here is the patch that seems to work. Using package imagemin-pngquant with version 9.0.0, look at index.js Change this code (literally just error.code to error.exitCode):

	const promise = subprocess
		.then(result => result.stdout) // eslint-disable-line promise/prefer-await-to-then
		.catch(error => {
			if (error.code === 99) {
				return input;
			}

			error.message = error.stderr || error.message;
			throw error;
		});

To this:

	const promise = subprocess
		.then(result => result.stdout) // eslint-disable-line promise/prefer-await-to-then
		.catch(error => {
			if (error.exitCode === 99) {
				return input;
			}

			error.message = error.stderr || error.message;
			throw error;
		});

I will leave some of the details here for others to verify… Chasing some issues in which the pngquant invocation returns an error code that in index.js shows as “undefined” (should be 99) so the imagemin-pngquant routine will realize it is ok to skip the particular attempt…running the binary manually always gets a 99. When this code happens, the input is returned as output (ie no change). So, the question is why does an error code of undefined appear sometimes?

I have been seeing the same issues since recent updates of sharp-related libraries. After convincing myself there were no cache issues, I then homed in on failing PNG handling. Then I thought it was transparent PNG only. In the end, I think there are timing issues in how long the executable runs before the subprocess exits. Finally, I realized the code in imagemin-pngquant was checking the wrong element (error.code). Once I changed to error.exitCode, everything worked.

For reference, here is my environment (using v9.0.0 of imagemin-pngquant), from gatsby info:

gatsby info
  System:
    OS: Linux 5.4 Ubuntu 20.04.1 LTS (Focal Fossa)
    CPU: (12) x64 Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
    Shell: 5.0.17 - /bin/bash

  Binaries:
    Node: 12.18.3 - ~/.nvm/versions/node/v12.18.3/bin/node
    npm: 6.14.8 - ~/.nvm/versions/node/v12.18.3/bin/npm
  Languages:
    Python: 2.7.18 - /usr/bin/python
  Browsers:
    Firefox: 80.0
  npmPackages:
    gatsby: ^2.24.53 => 2.24.53
    gatsby-background-image: ^1.1.2 => 1.1.2
    gatsby-image: ^2.4.16 => 2.4.16
    gatsby-plugin-catch-links: ^2.3.11 => 2.3.11
    gatsby-plugin-google-analytics: ^2.3.13 => 2.3.13
    gatsby-plugin-google-tagmanager: ^2.3.11 => 2.3.11
    gatsby-plugin-manifest: ^2.4.27 => 2.4.27
    gatsby-plugin-offline: ^3.2.26 => 3.2.26
    gatsby-plugin-react-helmet: ^3.3.10 => 3.3.10
    gatsby-plugin-sharp: ^2.6.31 => 2.6.31
    gatsby-remark-component: ^1.1.3 => 1.1.3
    gatsby-remark-copy-linked-files: ^2.3.13 => 2.3.13
    gatsby-remark-images: ^3.3.28 => 3.3.28
    gatsby-remark-relative-images: ^0.3.0 => 0.3.0
    gatsby-source-filesystem: ^2.3.27 => 2.3.27
    gatsby-transformer-remark: ^2.8.32 => 2.8.32
    gatsby-transformer-sharp: ^2.5.14 => 2.5.14
  npmGlobalPackages:
    gatsby-cli: 2.12.91
    gatsby-starter-default: 0.1.0

And a quick check shows pngquant itself is version 2.10.1 on my machine (pulled from the packages under node_modules/pngquant-bin)

I’ll sort out a pull request once I am sure there are no negative effects - doing the above at least allowed things to build and images to process.

BTW, this is the detailed “error” that results when running the pngquant binary manually (an error code 99):

Original error:
stdin:
  read 8KB file
  made histogram...693 colors found
  selecting colors...3%
  selecting colors...24%
  selecting colors...27%
  selecting colors...48%
  selecting colors...51%
  selecting colors...72%
  selecting colors...93%
  selecting colors...100%
  moving colormap towards local minimum
  image degradation MSE=0.049 (Q=99) exceeded limit of 0.000 (100)
  writing truecolor image to stdout
Skipped 1 file out of a total of 1 file.

Note - on that “error” above, pngquant is just saying “I can’t do this so use the input as output”. From command line, the return code is 99. Eventually I sorted this out as mentioned above.