invalid tile byte count when using writing tif file

When trying to write a black-white tif file (a tumor mask for breast cancer H&E slides) I am running into an error which I have been trying to solve for the last few hours, but to no avail. I was wondering if you would happen to know what could be the cause of this. I have also tried to solve it by replacing it with a conda installation, but with the same result.

• example tif file and code included below

Installation information

   linux: 5.8.14-arch1-1
   python: 3.8
   vips: https://github.com/libvips/libvips/releases/download/v8.11.2/vips-8.11.2.tar.gz
   libjpeg: libjpeg-turbo8-dev
   ibgtk2.0-dev
   libgsf-1-dev
   libtiff5-dev
   pyvips: 2.1.15

sample code:

mask_fname = 'tumor_010_mask.tif'
image = pyvips.Image.new_from_file(mask_fname)

# does not work, gives TIFFFillTile errors
arr = np.ndarray(buffer=image.write_to_memory())

# does not work, gives TIFFFillTile errors
image.write_to_file('test.tif')

Whenever I try to run the above sample code, I get the following errors:

---------------------------------------------------------------------------
Error                                     Traceback (most recent call last)
<ipython-input-29-21eb559af30f> in <module>
      1 image = pyvips.Image.new_from_file(mask_fname, access=pyvips.Access.SEQUENTIAL)
      2 
----> 3 arr = np.ndarray(buffer=image.write_to_memory(),
      4                            dtype=dict_inverse(dtype_to_format)[image.format],
      5                            shape=[image.height, image.width, image.bands])

/usr/local/lib/python3.8/site-packages/pyvips/vimage.py in write_to_memory(self)
    619         pointer = vips_lib.vips_image_write_to_memory(self.pointer, psize)
    620         if pointer == ffi.NULL:
--> 621             raise Error('unable to write to memory')
    622         pointer = ffi.gc(pointer, glib_lib.g_free)
    623 

Error: unable to write to memory
  TIFFFillTile: 0: Invalid tile byte count, tile 0
TIFFFillTile: 0: Invalid tile byte count, tile 1
TIFFFillTile: 0: Invalid tile byte count, tile 2
TIFFFillTile: 0: Invalid tile byte count, tile 3
TIFFFillTile: 0: Invalid tile byte count, tile 4
TIFFFillTile: 0: Invalid tile byte count, tile 5
TIFFFillTile: 0: Invalid tile byte count, tile 6
.
.
.
TIFFFillTile: 0: Invalid tile byte count, tile 189
TIFFFillTile: 0: Invalid tile byte count, tile 190

example.zip

[stephandooper]

Then it seems that something is going wrong within the ASAP pipeline. As a sanity check I have read an original tissue file into asap and then wrote it back to tif.

The result is that pyvips is able to read the resulting array into memory, and write it to tif without any problem. The only repeating problem is that the masks generated by the asap pipeline are able to be processed into pyvips.

As a workaround, I have been able to come up with a rather intensive workaround: I can read the culprit masks into memory using ASAP, and then read the resulting array into pyvips using read_from_buffer. Then, I can simply save the resulting array using pyvips. The resulting file will have no problem being handled by pyvips.

As for the problem itself: I suspect you are right in saying that it has something to do with ASAP. I will try to investigate this further and post an issue when I am able to reproduce the problem.

Since the problem probably lies somewhere within ASAP I will close this issue.

[jcupitt]

Sorry, I keep making very small posts. In pyvips you can create an image like this with:

Python 3.9.5 (default, May 11 2021, 08:20:37) 
[GCC 10.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pyvips
>>> x = pyvips.Image.new_from_file("Mirax2.2-1.mrxs")
>>> x.width
101832
>>> x.height
219976
>>> x.bands
4
>>> mask = x[1] > 128
>>> mask.write_to_file("x.tif", compression="lzw", tile_width=512, tile_height=512, pyramid=True)
>>>

The tile looks like this:

$ tiffinfo x.tif 
TIFF Directory at offset 0x173812d0 (389550800)
  Image Width: 101832 Image Length: 219976
  Tile Width: 512 Tile Length: 512
  Resolution: 42757.8, 42674.7 pixels/cm
  Bits/Sample: 8
  Sample Format: unsigned integer
  Compression Scheme: LZW
  Photometric Interpretation: min-is-black
  Orientation: row 0 top, col 0 lhs
  Samples/Pixel: 1
  Planar Configuration: single image plane
  Predictor: horizontal differencing 2 (0x2)
TIFF Directory at offset 0x246a3f1a (610942746)
  Subfile Type: reduced-resolution image (1 = 0x1)
  Image Width: 50916 Image Length: 109988
  Tile Width: 512 Tile Length: 512
  Resolution: 42757.8, 42674.7 pixels/cm
  Bits/Sample: 8
  Compression Scheme: LZW
  Photometric Interpretation: min-is-black
  Orientation: row 0 top, col 0 lhs
  Samples/Pixel: 1
  Planar Configuration: single image plane
  Predictor: horizontal differencing 2 (0x2)
.... etc.

And can be opened like this (reading page 7 to keep the size reasonable):

$ python3
Python 3.9.5 (default, May 11 2021, 08:20:37) 
[GCC 10.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pyvips
>>> x = pyvips.Image.new_from_file("x.tif", page=7)
>>> x.write_to_file("x.png")
>>> 

To make: