Skip to content

feat: Zarr export bridge for ensemble probabilistic forecasts and CRPS N=1 fix#545

Closed
ayulockedin wants to merge 1 commit intomllam:mainfrom
ayulockedin:feat/zarr-bridge-and-metrics-patch
Closed

feat: Zarr export bridge for ensemble probabilistic forecasts and CRPS N=1 fix#545
ayulockedin wants to merge 1 commit intomllam:mainfrom
ayulockedin:feat/zarr-bridge-and-metrics-patch

Conversation

@ayulockedin
Copy link
Copy Markdown

Describe your changes

Summary of changes:

  • Updated _create_dataarray_from_tensor in neural_lam/models/ar_model.py to detect and handle 4D probabilistic tensor outputs shaped (B, T, N, F).
  • Added ensemble_member dimension and coordinate mapping when constructing xarray DataArray objects for probabilistic outputs.
  • Preserved existing behavior for deterministic 2D and 3D tensor conversion paths.
  • Patched the N=1 fallback branch in crps_ens in neural_lam/metrics.py to forward sum_vars during mae delegation, preventing shape divergence when sum_vars=False.
  • Updated mae to safely handle pred_std=None for valid internal delegation.
  • Registered crps_ens in the metric registry for lookup compatibility.
  • Added unit tests in tests/test_metrics.py for:
  • crps_ens N=1 shape consistency with sum_vars=False
  • 4D tensor conversion with ensemble_member coordinate creation

Motivation and context:

  • As the codebase expands probabilistic forecasting support, unroll_prediction produces 4D outputs that include an ensemble axis.
  • The previous conversion bridge was deterministic-oriented and did not map this axis to a named ensemble_member coordinate, blocking clean downstream export and verification workflows.
  • This also fixes a real shape-consistency bug in the CRPS ensemble metric for single-member ensembles.

Dependencies:

  • xarray
  • pytest
  • No new external runtime dependency introduced by this change.

Issue Link

Type of change

  • 🐛 Bug fix (non-breaking change that fixes an issue)
  • ✨ New feature (non-breaking change that adds functionality)
  • 💥 Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • 📖 Documentation (Addition or improvements to documentation)

Checklist before requesting a review

  • My branch is up-to-date with the target branch.
  • I have performed a self-review of my code.
  • For new or modified functions/classes, I added docstrings describing purpose, inputs, and outputs.
  • I added in-line comments where intent was not obvious.
  • I updated the README to cover introduced code changes.
  • I added tests proving the bug fix and feature behavior.
  • I used an imperative PR title.
  • I requested a reviewer and an assignee (or tagged maintainers, depending on repo permissions).

Checklist for reviewers

  • The code is readable.
  • The code is well tested.
  • The code is documented (including parameters and return values).
  • The code is maintainable.

Author checklist after completed review

  • I added a CHANGELOG entry reflecting this change type.

CHANGELOG category guidance:

  • added: new functionality
  • changed: behavior changes
  • fixed: bug fixes
  • maintenance: repository maintenance, CI/CD, docs

Checklist for assignee

  • PR is up to date with the base branch.
  • Tests pass.
  • If not maintenance/bugfix-only, the PR is assigned to a milestone.
  • CHANGELOG entry is present with the correct category.
  • PR is squashed and merged when ready.

@kshirajahere
Copy link
Copy Markdown
Contributor

duplicate of #521

@ayulockedin
Copy link
Copy Markdown
Author

ayulockedin commented Mar 29, 2026

I did see #521 but i plan to evaluate from here this was just a anchor regarding the issue and i further plan on integrating the energy score metric but i will close this pr to keep the queue clean

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants