Skip to content

feat: Add batched tensor expansion utilities for O(S) ensemble unroll…#489

Open
Debadri-das wants to merge 3 commits intomllam:mainfrom
Debadri-das:feat/batched-ensemble-utils
Open

feat: Add batched tensor expansion utilities for O(S) ensemble unroll…#489
Debadri-das wants to merge 3 commits intomllam:mainfrom
Debadri-das:feat/batched-ensemble-utils

Conversation

@Debadri-das
Copy link
Copy Markdown

Describe your changes

This PR introduces a standalone batch-expanding PyTorch utility (expand_ensemble_batch and fold_ensemble_batch) to optimize tensor manipulation for Ensemble Lateral Boundary Conditions (LBCs) and probabilistic forecasting variables.

Motivation and Context: Currently, probabilistic processing of $S$ ensemble members often relies on sequential for loops. To support the upcoming model refactoring and probabilistic capabilities without introducing $O(S)$ sequential bottlenecks, these functions leverage torch.repeat_interleave() and .view() to dynamically flatten state tensors (B, ...) -> (B * S, ...). This enables optimized parallelized operations over the GPU without permanently mutating standard AR model dimensions.

Dependencies: None. Built strictly with standard PyTorch tensor primitives.

Issue Link

#49 and #62

Type of change

  • 🐛 Bug fix (non-breaking change that fixes an issue)
  • ✨ New feature (non-breaking change that adds functionality)
  • 💥 Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • 📖 Documentation (Addition or improvements to documentation)

Checklist before requesting a review

  • My branch is up-to-date with the target branch .
  • I have performed a self-review of my code
  • For any new/modified functions/classes I have added docstrings that clearly describe its purpose, expected inputs and returned values
  • I have placed in-line comments to clarify the intent of any hard-to-understand passages of my code
  • I have updated the README to cover introduced code changes
  • I have added tests that prove my fix is effective or that my feature works
  • I have given the PR a name that clearly describes the change, written in imperative form (context).
  • I have requested a reviewer and an assignee (assignee is responsible for merging). This applies only if you have write access to the repo, otherwise feel free to tag a maintainer to add a reviewer and assignee.

Checklist for reviewers

Each PR comes with its own improvements and flaws. The reviewer should check the following:

  • the code is readable
  • the code is well tested
  • the code is documented (including return types and parameters)
  • the code is easy to maintain

Author checklist after completed review

  • I have added a line to the CHANGELOG describing this change, in a section reflecting type of change (add section where missing):
    • added: when you have added new functionality
    • changed: when default behaviour of the code has been changed
    • fixes: when your contribution fixes a bug
    • maintenance: when your contribution is relates to repo maintenance, e.g. CI/CD or documentation

Checklist for assignee

  • PR is up to date with the base branch
  • the tests pass
  • (if the PR is not just maintenance/bugfix) the PR is assigned to the next milestone. If it is not, propose it for a future milestone.
  • author has added an entry to the changelog (and designated the change as added, changed, fixed or maintenance)
  • Once the PR is ready to be merged, squash commits and merge the PR.

Copy link
Copy Markdown
Contributor

@Sir-Sloth-The-Lazy Sir-Sloth-The-Lazy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a heads up you missed the case when S == T . Love the idea !

Copy link
Copy Markdown
Contributor

@Sir-Sloth-The-Lazy Sir-Sloth-The-Lazy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this intentional ?

@sadamov
Copy link
Copy Markdown
Collaborator

sadamov commented Mar 23, 2026

linking #226 and #332 here, to remember the potential connection. this is one for @joeloskarsson to review.

@sadamov sadamov requested a review from joeloskarsson March 23, 2026 05:15
@sadamov sadamov added the enhancement New feature or request label Mar 23, 2026
@Debadri-das
Copy link
Copy Markdown
Author

is this intentional ?
No, the code was not intentional at all, it was a flaw in the design, which I fixed now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants