Improve ImageJ's handling of large (>4GB) TIFF files

It should be possible to quickly load, browse and save very large TIFF files in a coherent way. There are several pieces to this:

• Ensure SCIFIO transparently writes to BigTIFF when exporting using File > Export > Image…
• Make SCIFIO able to import ImageJ1’s “raw pseudo TIFF” format
• Make SCIFIO’s “virtual stack” (cell image) performance as good as possible
• Make SCIFIO recognize signs (e.g., an ImageJ1-style ImageDescription in the first IFD) when a TIFF is written monotonically, so it can be parsed more quickly (i.e., without explicitly reading all the IFDs)
• Make SCIFIO add such signs (probably some new metadata as a directory entry in the first IFD) when writing TIFFs

Also critical is to improve performance of Bio-Formats setId calls of various high-impact proprietary file formats such as ND2, although that work is outside the scope of this issue.

See also these mailing list threads:

• Problems with Saving Tiff greater than 4 GB & bigTiff exporter
• Processing huge files
• bfconverter nd2 to tif

[ctrueden]

The only problem with ICS is that it will lose a lot of metadata compared to OME-TIFF. So we do really need to fix these limitations with respect to 4+GB TIFF & OME-TIFF files.

[tischi]

to me ICS seems to be the only standard worth talking of at the moment. works with Imaris, ImageJ, Arivis, Matlab, Python, anything that uses BioFormats anyway.