bug reading grayscale images using Pillow 9.1

Pillow 9.1 was released today and has broken a few of the scikit-image tests.

The traceback shows the issue being raised from _palette_is_grayscale within imageio’s pil_legacy.py.

The failure can be reproduced with:

from skimage import data
img = data.eagle()

Here is a traceback from one of the failed CI runs where the environment used Pillow 9.1.0 and imageio 2.16.1:

================================== FAILURES ===================================
_________________________________ test_eagle __________________________________

    def test_eagle():
    return imread(_fetch(f), as_gray=as_gray)
C:\hostedtoolcache\windows\Python\3.8.10\x64\lib\site-packages\skimage\io\_io.py:53: in imread
    img = call_plugin('imread', fname, plugin=plugin, **plugin_args)
C:\hostedtoolcache\windows\Python\3.8.10\x64\lib\site-packages\skimage\io\manage_plugins.py:207: in call_plugin
    return func(*args, **kwargs)
C:\hostedtoolcache\windows\Python\3.8.10\x64\lib\site-packages\skimage\io\_plugins\imageio_plugin.py:10: in imread
    return np.asarray(imageio_imread(*args, **kwargs))
C:\hostedtoolcache\windows\Python\3.8.10\x64\lib\site-packages\imageio\__init__.py:86: in imread
    return imread_v2(uri, format=format, **kwargs)
C:\hostedtoolcache\windows\Python\3.8.10\x64\lib\site-packages\imageio\v2.py:160: in imread
    return file.read(index=0, **kwargs)
C:\hostedtoolcache\windows\Python\3.8.10\x64\lib\site-packages\imageio\core\legacy_plugin_wrapper.py:132: in read
    reader = self.legacy_get_reader(**kwargs)
C:\hostedtoolcache\windows\Python\3.8.10\x64\lib\site-packages\imageio\core\legacy_plugin_wrapper.py:104: in legacy_get_reader
    return self._format.get_reader(self._request)
C:\hostedtoolcache\windows\Python\3.8.10\x64\lib\site-packages\imageio\core\format.py:190: in get_reader
    return self.Reader(self, request)
C:\hostedtoolcache\windows\Python\3.8.10\x64\lib\site-packages\imageio\core\format.py:281: in __init__
    self._open(**self.request.kwargs.copy())
C:\hostedtoolcache\windows\Python\3.8.10\x64\lib\site-packages\imageio\plugins\pillow_legacy.py:394: in _open
    return PillowFormat.Reader._open(self, pilmode=pilmode, as_gray=as_gray)
C:\hostedtoolcache\windows\Python\3.8.10\x64\lib\site-packages\imageio\plugins\pillow_legacy.py:303: in _open
    as_gray=as_gray, is_gray=_palette_is_grayscale(self._im)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

pil_image = <PIL.PngImagePlugin.PngImageFile image mode=P size=1826x2019 at 0x29176FBD8E0>

    def _palette_is_grayscale(pil_image):
        if pil_image.mode != "P":
            return False
        elif pil_image.info.get("transparency", None):  # see issue #475
            return False
        # get palette as an array with R, G, B columns
>       palette = np.asarray(pil_image.getpalette()).reshape((256, 3))
E       ValueError: cannot reshape array of size 765 into shape (256,3)

C:\hostedtoolcache\windows\Python\3.8.10\x64\lib\site-packages\imageio\plugins\pillow_legacy.py:674: ValueError

[hmaarrfk]

https://gitlab.com/scikit-image/data/-/blob/master/eagle.png

[FirefoxMetzger]

requires imageio >= 2.16?

@grlee77 Yes, in 2.16 we added an explicit namespace for the upcoming API so that we can make it available for those who wish to explore it without disrupting existing workflows. The docs for it are here: https://imageio.readthedocs.io/en/latest/reference/index.html

it looks like pillow.py -> pillow_legacy.py appeared earlier than that in the repo, though

@grlee77 Also yes, because we were/are building the new API and new plugins incrementally. So there is a new (I think better) pillow plugin which will eventually replace the old one, but it is currently only available through the v3 API until I finish writing some more plumbing code.

---

Regarding the present issue; it seems that starting with pillow 9.1 pillow now allows pallets to contain less than 256 entries (at least for paletted PNG). pillow_legacy.py assumes that palettes have 256 entries (which was hardcoded) and hence breaks when pillow returns less. eagle.png has 255 colors in the palette which is why things broke. This means that the fix is a one-liner.

At the same time I saw that some of our unit tests broke when I switched to pillow 9.1, so I will have a look at those as well and fix them all in one PR.

Edit: Done. If you want, @grlee77 you can check out the PR linked below and see if that resolves the problem on your end as well.