Skip to content

Fix stale CRS metadata after reprojection #237

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
taufik-shf wants to merge 4 commits into
base: develop
Choose a base branch
from
Open

Fix stale CRS metadata after reprojection #237

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
cec3167
ensure projection metadata is JSON serializable
taufik-shf Jan 30, 2026
fe2b5e5
loosen datacube requirement and pin odc-stac
taufik-shf Jan 30, 2026
49a935f
restore CRS after xr_reproject and geomedian_with_mads
taufik-shf Feb 3, 2026
dcbc210
Apply pre-commit fixes
taufik-shf Feb 4, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion odc/stats/io.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@

from odc.geo.geobox import GeoBox
from odc.geo.geobox import pad as gbox_pad
from odc.geo.xr import xr_reproject
from odc.geo.xr import xr_reproject, assign_crs

from ._grouper import group_by_nothing, solar_offset
from odc.algo._masking import (
Expand Down Expand Up @@ -814,6 +814,9 @@ def load_with_native_transform(
**extra_args,
)

# Ensure output advertises the destination CRS consistently
Copy link
Copy Markdown
Contributor

@pjonsson pjonsson Feb 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't the output from xr_reproject have the right CRS, so this issue should be fixed inside that function instead?

Copy link
Copy Markdown
Author

@taufik-shf taufik-shf Feb 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right - it should be fixed in xr_reproject().

I'm happy to attempt a fix in odc-geo if you'd prefer, otherwise we could merge this workaround for now and I'll open an issue upstream.

Copy link
Copy Markdown
Contributor

@pjonsson pjonsson Feb 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Without speaking for this repository, I think it would make sense to try to fix the problems in odc-algo and odc-geo. It sounds like you've found some bugs, and if I'm wrong and the methods are meant to behave the way they currently behave, you're more likely to get that feedback if you make PRs to those repositories.

_yy = assign_crs(_yy, crs=geobox.crs)

if isinstance(_yy, xr.DataArray) and vars_to_scale:
_yy = _yy > 64
elif vars_to_scale:
Expand Down
7 changes: 5 additions & 2 deletions odc/stats/model.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -485,7 +485,7 @@ def render_assembler_metadata(

return dataset_assembler

def render_metadata(
def render_metadata( # pylint: disable=too-many-locals
self,
ext: str = EXT_TIFF,
processing_dt: datetime | None = None,
Expand Down Expand Up @@ -528,8 +528,11 @@ def render_metadata(
)
ProjectionExtension.add_to(item)
proj_ext = ProjectionExtension.ext(item)

proj_ext.apply(
epsg=geobox.crs.epsg, transform=geobox.transform, shape=list(geobox.shape)
epsg=geobox.crs.epsg,
transform=list(geobox.transform), # ensure transform is JSON serializable
shape=list(geobox.shape),
)

# Lineage last
Expand Down
16 changes: 16 additions & 0 deletions odc/stats/plugins/gm.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
from ._registry import StatsPluginInterface, register
from odc.algo import enum_to_bool, erase_bad
from odc.algo import mask_cleanup
from odc.geo.xr import assign_crs
import logging

_log = logging.getLogger(__name__)
Expand Down Expand Up @@ -123,6 +124,21 @@ def reduce(self, xx: xr.Dataset) -> xr.Dataset:
gm = geomedian_with_mads(xx, **cfg)
gm = gm.rename(self._renames)

# geomedian_with_mads drops spatial_ref; re-attach from input
Copy link
Copy Markdown
Contributor

@pjonsson pjonsson Feb 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same question here, shouldn't geomedian_with_mads return an object with the right metadata?

Copy link
Copy Markdown
Author

@taufik-shf taufik-shf Feb 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, it should be fixed in geomedian_with_mads().

I can try to fix it in odc-algo if you'd prefer, or we can merge this workaround for now until the upstream is fixed. Once it is fixed there, I'll remove the workaround to keep the code clean.

Copy link
Copy Markdown
Member

@Kirill888 Kirill888 Mar 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

see opendatacube/odc-geo#259 (comment)

crs = getattr(xx.odc, "crs", None) # type: ignore[attr-defined]
if crs is not None:
gm = gm.copy()

# Remove stale CRS metadata
gm.attrs.pop("crs", None)
gm.attrs.pop("grid_mapping", None)
for v in gm.data_vars:
gm[v].attrs.pop("crs", None)
gm[v].attrs.pop("grid_mapping", None)

gm = gm.drop_vars(["spatial_ref", "crs"], errors="ignore")
gm = assign_crs(gm, crs=crs)

return gm


Expand Down
3 changes: 2 additions & 1 deletion setup.cfg
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,8 @@ install_requires =
botocore
click>=8.0.0
dask
datacube==1.9.5
datacube>=1.9.5
Copy link
Copy Markdown
Contributor

@pjonsson pjonsson Feb 4, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The datacube version is pinned because memory consumption balloons for odc-stats with later versions. I'm not involved in odc-stats so I don't know about the timeline for resolving that.

Copy link
Copy Markdown
Author

@taufik-shf taufik-shf Feb 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the context! I wasn't aware of the memory issue with later datacube versions.

I made this change because my environment uses datacube 1.9.10, and the strict pin was causing dependency conflicts for me.

Would it be possible to loosen the pin to allow users to manage their own datacube version if needed?

odc-stac==0.4.0
distributed>=2025.4,<2025.9
numpy
odc-cloud[ASYNC]>=0.2.5
Expand Down