Skip to content

to_neural_lam() to wmg.save for neural-lam tensor-on-disk format#123

Open
prajwal-tech07 wants to merge 1 commit intomllam:mainfrom
prajwal-tech07:issue-384/to-neural-lam
Open

to_neural_lam() to wmg.save for neural-lam tensor-on-disk format#123
prajwal-tech07 wants to merge 1 commit intomllam:mainfrom
prajwal-tech07:issue-384/to-neural-lam

Conversation

@prajwal-tech07
Copy link
Copy Markdown

Describe your changes

Add a to_neural_lam() function to wmg.save that serializes graph components (from wmg.create.archetype.*(..., return_components=True)) to the tensor-on-disk .pt format expected by neural_lam.utils.load_graph().

This is the first step in bridging weather-model-graphs and neural-lam — putting the serialization logic in wmg so that neural-lam's CLI can remain a thin wrapper.

What it does:

  • Accepts the {"g2m", "m2m", "m2g"} dict returned by any archetype with return_components=True
  • Writes 7 core .pt files (g2m/m2g as single tensors, m2m/mesh_features as lists)
  • Writes 4 additional files for hierarchical graphs (mesh_up/mesh_down edge_index and features)
  • Edge features are written raw (unnormalized) — neural-lam normalizes at load time
  • Mesh node positions are pre-normalized by max(abs(pos)), matching the existing neural-lam convention

Supports all three archetypes:

  • Keisler (flat single-scale)
  • GraphCast (flat multiscale)
  • Oskarsson (hierarchical)

Also adds two internal helpers (_graph_to_edge_tensors, _graph_to_node_features) that cleanly strip heterogeneous node/edge attributes before PyG conversion.

Tests: 22 new tests across 4 test classes covering all archetypes, shape/dtype validation, normalization checks, and edge cases.

No new dependencies — uses the existing [pytorch] optional extras (torch, torch-geometric).

Issue Link

Refs mllam/neural-lam#384

Type of change

  • 🐛 Bug fix (non-breaking change that fixes an issue)
  • ✨ New feature (non-breaking change that adds functionality)
  • 💥 Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • 📖 Documentation (Addition or improvements to documentation)

Checklist before requesting a review

  • My branch is up-to-date with the target branch - if not update your fork with the changes from the target branch (use pull with --rebase option if possible).
  • I have performed a self-review of my code
  • For any new/modified functions/classes I have added docstrings that clearly describe its purpose, expected inputs and returned values
  • I have placed in-line comments to clarify the intent of any hard-to-understand passages of my code
  • I have updated the documentation to cover introduced code changes
  • I have added tests that prove my fix is effective or that my feature works
  • I have given the PR a name that clearly describes the change, written in imperative form (context).
  • I have requested a reviewer and an assignee (assignee is responsible for merging)

Checklist for reviewers

Each PR comes with its own improvements and flaws. The reviewer should check the following:

  • the code is readable
  • the code is well tested
  • the code is documented (including return types and parameters)
  • the code is easy to maintain

Author checklist after completed review

  • I have added a line to the CHANGELOG describing this change, in a section
    reflecting type of change (add section where missing):
    • added: when you have added new functionality
    • changed: when default behaviour of the code has been changed
    • fixes: when your contribution fixes a bug

Checklist for assignee

  • PR is up to date with the base branch
  • the tests pass
  • author has added an entry to the changelog (and designated the change as added, changed or fixed)
  • Once the PR is ready to be merged, squash commits and merge the PR.

…ormat

Add 	o_neural_lam() function that saves graph components to the
tensor-on-disk format expected by
eural_lam.utils.load_graph().

Supports all three archetypes:
- Keisler (flat single-scale)
- GraphCast (flat multiscale)
- Oskarsson (hierarchical)

Output files:
- g2m/m2g: single tensors (edge_index, features)
- m2m: lists of tensors per level
- mesh_features: position-normalized node features per level
- mesh_up/mesh_down: hierarchical inter-level edges (when hierarchical=True)

Edge features are written raw (unnormalized) - neural-lam normalizes at
load time. Mesh node positions are pre-normalized by max(abs(pos)).

Includes comprehensive tests covering all 3 archetypes, shape validation,
dtype checks, normalization verification, and edge cases.

Refs: mllam/neural-lam#384
@prajwal-tech07 prajwal-tech07 force-pushed the issue-384/to-neural-lam branch from 88fdd98 to c2e7b5e Compare March 28, 2026 18:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant