Read Geotiff: java.lang.RuntimeException: javax.imageio.IIOException: 64-bit samples are not supported for Horizontal differencing Predictor

[jornfranke]

Expected behavior

Geotiff can read without "java.lang.RuntimeException: javax.imageio.IIOException: 64-bit samples are not supported for Horizontal differencing Predictor".

Actual behavior

I receive for certain geotiff files "java.lang.RuntimeException: javax.imageio.IIOException: 64-bit samples are not supported for Horizontal differencing Predictor".

I digged a bit into this - it seems because the geotiff file is compressed (DEFLATE) using QGIS (GDAL).
However, from the error I get that certain predictor settings for deflate are not supported (cf. also: https://kokoalberti.com/articles/geotiff-compression-optimization-guide/). I still have to try using no predictor. It may work then (I have to check the QGIS GDAL settings if they use by default horizontal predictor).

I looked also in the Sedona code and in theory it should support it (gc-geotiff dependency
). It is a bit more difficult to troubleshoot, because this depends on javax.media.jai_imageio and it.geosolutions.imageio-ext.imageio-ext-tiff.

It seems javax.media.jai_imageio in principle supports tiff deflate compression (https://github.com/jai-imageio/jai-imageio-core/blob/master/src/main/java/com/github/jaiimageio/plugins/tiff/BaselineTIFFTagSet.java#L249).

However, it.geosolutions.imageio-ext.imageio-ext-tiff (https://github.com/geosolutions-it/imageio-ext/tree/master/plugin/tiff) has an optimized tiff format reader/writer which may support less.

The "javax.media.jai_imageio" is a bit special due to its "special history" and also different variant exists (e.g. https://github.com/senbox-org/jai-imageio-core/blob/master/src/share/classes/com/sun/media/imageioimpl/plugins/tiff/TIFFDeflateDecompressor.java).

I saw that it seems to be also integrated back in JDK9 ("Note that most of these formats are now supported by Java 9")

However gc-geotools seem to use the maven package (cf. https://github.com/geotools/geotools/blob/main/modules/plugin/geotiff/pom.xml#L105), which is based on an outdated version (https://github.com/jai-imageio/jai-imageio-core). Nevertheless it may not make sense to use the JDK version, because it is in the JDK module "java.desktop", which may not be available on server versions. Probably that is why the extensions it.geosolutions.imageio-ext.imageio-ext-tiff are included in Sedona?

I know this is probably not a Sedona problem, but a problem that Sedona users may face - so I can propose to add a documentation page in Sedona to reflect this (which compressions are supported etc.).

Steps to reproduce the problem

I took a small file from https://www.ipcc-data.org/ to reproduce the problem. The original file "ipcc_eu_CDD_v_2040_r45_a.tiff" is not compressed and works perfectly fine with Sedona.

The other file is a cropped version of the original file of the Austria region but with DEFLATE compression using standard QGIS GDAL settings (probably horizontal predictor is default in QGIS (3.28.1), I have to check, still have to see if this works if I use DEFLATE with no predictor).

ipcc_.tar.gz

You can reproduce it easily by using the example provided in the documentation for RS_Values, which uses RS_FromGeoTiff

Settings

Sedona version = 1.4.1

Apache Spark version = 3.2.0

Apache Flink version = N/A

API type = Python?

Scala version =2.12

JRE version = 1.8

Python version = 3.9

Environment = CDP

[Kontinuation]

GeoTools uses the GeoTiff reader provided by imageio-ext. As far as I know, this is the most comprehensive TIFF reader amongst all the ImageIO readers. I've tried using imageio shipped with temurin-11 and found that it cannot read neither attached image.

Horizontal differential predictor support for 32-bit data was introduced two years ago by commit geosolutions-it/imageio-ext@8707591 , we can submit a issue for 64-bit support.