RDD reproject: GeoAttrsError

[jdries]

Describe the bug

Original source: Open-EO/openeo-geopyspark-driver#984
This method call:

geotrellis/spark/src/main/scala/geotrellis/spark/reproject/TileRDDReproject.scala

Line 72 in 7960ea1

val sourceDataGridExtent = metadata.layout.createAlignedGridExtent(metadata.extent)

Can result in:
geotrellis.raster.GeoAttrsError: invalid cols: 0

in case of very specific inputs:

• an input cube with a small extent, like 1x1 pixel
• high latitudes, where square pixels in EPSG:4326 are basically rectangular pixels in UTM

Stack trace:

geotrellis.raster.GeoAttrsError: invalid cols: 0

at geotrellis.raster.GridExtent.<init>(GridExtent.scala:45)
at geotrellis.raster.GridExtent.<init>(GridExtent.scala:58)
at geotrellis.raster.reproject.ReprojectRasterExtent$.apply(ReprojectRasterExtent.scala:71)
at geotrellis.raster.reproject.ReprojectRasterExtent$.apply(ReprojectRasterExtent.scala:82)
at geotrellis.raster.reproject.ReprojectRasterExtent$.apply(ReprojectRasterExtent.scala:86)
at geotrellis.spark.reproject.TileRDDReproject$.apply(TileRDDReproject.scala:73)
at geotrellis.spark.reproject.TileRDDReproject$.apply(TileRDDReproject.scala:285)
at geotrellis.spark.reproject.TileRDDReprojectMethods.reproject(TileRDDReprojectMethods.scala:92)
at org.openeo.geotrellis.OpenEOProcessesSpec.testEdgeCase(OpenEOProcessesSpec.scala:668)

To Reproduce

This example shows the full reprojection case and throws the error:

import geotrellis.spark._
import geotrellis.spark.reproject._

val layout = LayoutDefinition(RasterExtent(Extent(-120.550992, 71.82628788888888, -120.5421031111111, 71.83517677777778),32,32),32,32)
val targetLayout = LayoutDefinition(RasterExtent(Extent(585180.0, 7971640.0, 585820.0, 7972280.0),32,32),32,32)
val extent = Extent(-120.550992, 71.834899, -120.550792, 71.835099)
val cube: TileLayerRDD[SpatialKey] = TileLayerRDDBuilders.createTileLayerRDD(OpenEOProcessesSpec.sc,ByteArrayTile.fill(1,32,32),layout.tileLayout,LatLng)

val utm = CRS.fromEpsgCode(32610)

val badMetadata = cube.metadata.copy(crs = LatLng, extent = extent,layout = layout)
ContextRDD(cube,badMetadata).reproject(utm, targetLayout, Reproject.Options.DEFAULT)

The code below shows inputs to TileRDDReproject and the specific call to ReprojectRasterExtent which subsequently fails:

val layout = LayoutDefinition(RasterExtent(Extent(-120.550992, 71.82628788888888, -120.5421031111111, 71.83517677777778),32,32),32,32)
val extent = Extent(-120.550992, 71.834899, -120.550792, 71.835099)
var sourceDataGridExtent = layout.createAlignedGridExtent(extent)
val utm = CRS.fromEpsgCode(32610)
ReprojectRasterExtent(sourceDataGridExtent, LatLng, utm)

Note: I'm also looking into a test case at the level of TileRDDReproject...

Expected behavior

While this is clearly an edge case, it seems that it could be solved easily in the case where
targetLayout: Either[LayoutScheme, LayoutDefinition],
is actually a 'LayoutDefinition'.
In that case, the call to ReprojectRasterExtent is not needed at all, and we can compute a target extent that is aligned with the layout definition.

More specifically something like this:

val (targetCellSize:CellSize,targetDataExtent:Extent)=
    targetLayout match {
      case Right(l) => {
        (l.cellSize, l.createAlignedGridExtent(ProjectedExtent(metadata.extent,metadata.crs).reproject(destCrs)).extent)
      }
      case Left(l) => {
        val passthroughGridExtent = ReprojectRasterExtent(sourceDataGridExtent, metadata.crs, destCrs)
        (passthroughGridExtent.cellSize,passthroughGridExtent.extent)
      }
    }

Environment

any

[pomadchin]

Thanks for reporting! Yes, sounds like a not convenient edge case.

[jdries]

Thanks, I'm looking into creating a PR myself.

[pomadchin]

Thank you! Also I wonder that in some of these edge cases a slim math solution may work 🤔 That's not the first time smth like that arises.

[jdries]

Indeed, there are in fact options at this level:

geotrellis/raster/src/main/scala/geotrellis/raster/reproject/ReprojectRasterExtent.scala

Line 63 in 7960ea1

val newCols = (newExtent.width / pixelSizeX + 0.5).toLong

The problem occurs if the new extent width or height is smaller than 0.5 * pixelSize

In that case however, there's alternatives to throwing an exception:
A. if target cellsize is not set:
A.1 use non-square pixels, set X or Y pixel size so that it matches target extent width/height, resulting in exactly one pixel
A.2 use square pixel, but set it to min(targetExtent.width, targetExtent.height)
B. If target cellsize is set:
B.1 Still throw the error in this case
B.2 Return a gridextent with correct cellsize, but set the extent accordingly.

The solution I'll be testing first is at the level of TileRDDReproject and focuses on the case where a LayoutDefinition is available, I really think we can go for a more robust way of constructing an aligned target extent in that case.

[jdries]

We're hitting another variant of this exception. Also here, we only need the resolution and not the GridExtent, which causes the failure.
In this case, we are indeed loading very low resolution data, so indeed the output gridextent would be less than 0.5 pixels.

Caused by: geotrellis.raster.GeoAttrsError: invalid rows: 0
	at geotrellis.raster.GridExtent.<init>(GridExtent.scala:46)
	at geotrellis.raster.GridExtent.<init>(GridExtent.scala:58)
	at geotrellis.raster.reproject.ReprojectRasterExtent$.apply(ReprojectRasterExtent.scala:71)
	at geotrellis.raster.reproject.ReprojectRasterExtent$.apply(ReprojectRasterExtent.scala:76)
	at geotrellis.raster.geotiff.GeoTiffReprojectRasterSource.closestTiffOverview$lzycompute(GeoTiffReprojectRasterSource.scala:86)
	at geotrellis.raster.geotiff.GeoTiffReprojectRasterSource.closestTiffOverview(GeoTiffReprojectRasterSource.scala:81)
	at geotrellis.raster.geotiff.GeoTiffReprojectRasterSource.readBounds(GeoTiffReprojectRasterSource.scala:110)
	at geotrellis.raster.geotiff.GeoTiffReprojectRasterSource.read(GeoTiffReprojectRasterSource.scala:98)

[pomadchin]

@jdries is it captured in the PR? Apologizes for the long review times, will take a look later this week!

[jdries]

I created a second PR that addresses the problem more broadly. It basically implements option A1 above, trying to make a minimal change that preserves existing behaviour as much as possible.