[Question] What does threshold mean in a method toMatchSnapshot

Hi all, Now I am using jest as a test runner with jest-playwright-preset and jest-image-snapshot for image comparisons. Versions: jest: 27.0.4 jest-playwright-preset: 1.7.0 jest-image-snapshot: 4.5.1 playwright: 1.16.2

Screenshot comparisons looks like:

const customConfig = { threshold: 0.5 };
  expect(image).toMatchImageSnapshot({
    customDiffConfig: customConfig,
  });

Where threshold is the ratio of differing to the total number of pixels. Or the maximum value of differing pixels, in case if threshold is > 1.

When I started migrating to the native playwright runner, I faced with the image comparison issue. The expected and actual screenshots are not equals. But the difference is only a few pixels. For the image with resolution 1920x1080. I tried to set the threshold 0.5 or even more. Without any positive results.

I started looking the pixlematch source and find out that threshold determines how much each pixel can differ in color, and not the number of different pixels. https://github.com/mapbox/pixelmatch/blob/b9261a447515f5aff37a15cfab9f4a491868f720/index.js#L45

Further, the number of different pixels is simply compared with zeros in playwright matcher. https://github.com/microsoft/playwright/blob/2a0a44bc09e66ac4aaf03efb1b9d12ea27fd641f/packages/playwright-test/src/matchers/golden.ts#L69

In my case, this approach leads to failures even for several pixels with a fairly high threshold.

[dimkin-eu]

old one ticket, where pics with big and small diffs are same after same threshold https://github.com/microsoft/playwright/issues/9444

maybe its worth to investigate usage of some alternative?

[turnkey-commerce]

Everybody: We just landed two new options to toMatchSnapshot: pixelCount and pixelRatio. With these, you can have the following options to configure snapshot matching:

• threshold <[float]> an acceptable percieved color difference in the YIQ color space between pixels in compared images, between zero (strict) and one (lax). Defaults to 0.2.
• pixelCount <[int]> an acceptable amount of pixels that could be different, unset by default.
• pixelRatio <[float]> an acceptable ratio of pixels that are different to the total amount of pixels, between 0 and 1 , unset by default.

You can give them a try using npm i @playwright/test@next. Let us know how it goes!

@aslushnikov I gave them a try on a scenario and they work well for me, thanks very much. I found the options to actually be named as:

• maxDiffPixelRatio
• maxDiffPixels