Skip to content

feat: Implement Graph Specification Validator (Gap 6)#132

Open
Prince637-boo wants to merge 3 commits into
mllam:mainfrom
Prince637-boo:feat/gap6-graph-validator
Open

feat: Implement Graph Specification Validator (Gap 6)#132
Prince637-boo wants to merge 3 commits into
mllam:mainfrom
Prince637-boo:feat/gap6-graph-validator

Conversation

@Prince637-boo
Copy link
Copy Markdown

@Prince637-boo Prince637-boo commented Apr 3, 2026

Describe your changes

This PR introduces a Graph Structural & Geometric Validator for weather-model-graphs. While neural-lam#323 focuses on validating the "on-disk" format (shapes, types, and file structure) for model consumption, this validator focuses on the mathematical and geometric integrity of the graph during or after construction.

Key features added:

  • Geometric Consistency Check: Validates that 3D Cartesian coordinates are correctly normalized ($L^2 \approx 1.0$) to prevent projection errors.
  • Hierarchical Symmetry Check: Ensures that "up" (fine-to-coarse) edges are the exact reverse of "down" (coarse-to-fine) edges, maintaining bidirectional message-passing integrity.
  • Orphan Node Detection: Identifies grid nodes that are not connected to the mesh, which would cause silent failures during training.
  • Metadata Alignment: Cross-references the .pt tensors with the metadata.json to ensure consistency.

Issue Link

Relates to the "Gap 6" (Graph Validation) roadmap and supports the ongoing integration efforts in neural-lam#473.

Type of change

  • 🐛 Bug fix (non-breaking change that fixes an issue)
  • ✨ New feature (non-breaking change that adds functionality)
  • 💥 Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • 📖 Documentation (Addition or improvements to documentation)

Checklist before requesting a review

  • My branch is up-to-date with the target branch.
  • I have performed a self-review of my code.
  • For any new/modified functions/classes I have added docstrings.
  • I have placed in-line comments to clarify the intent.
  • I have updated the documentation to cover introduced code changes.
  • I have added tests that prove my feature works.
  • I have given the PR a name that clearly describes the change (imperative form).
  • I have requested a reviewer and an assignee.

Checklist for reviewers

  • The code is readable.
  • The code is well tested.
  • The code is documented (including return types and parameters).
  • The code is easy to maintain.

Author checklist after completed review

  • I have added a line to the CHANGELOG describing this change.
  • Section 'added': for new functionality.
  • Section 'changed': for default behaviour changes.
  • Section 'fixes': for bug fixes.

Checklist for assignee

  • PR is up to date with the base branch.
  • The tests pass.
  • Author has added an entry to the changelog (added, changed or fixed).
  • Once ready, squash commits and merge the PR.

@Joltsy10
Copy link
Copy Markdown
Contributor

Joltsy10 commented Apr 5, 2026

A few things:

  1. The discussion for a global validator is still quite early, given we dont even have a complete sense of how the global graph will end up/ what we will expect on the neural-lam side
  2. I believe the validator would exist on neural-lam and not wmg? The LAM validator spec is on the neural-lam repo, and the validator exists to validate the graphs that will be used in the models on the neural-lam repo

On the actual validator:

  1. The geometric consistency check is quite useful but we currently landed on passing lat/lon positions and not 3D cartesian (Hence why we need to have a proper discussion on what is required).
  2. The metadata.json is good as well but again it does not make the most sense to have a validator before a proper graph yeah? It would later become changing the graph architecture on the validator rather than the other way around

I'm not trying to discourage this, this looks really good but a lot of stuff isnt solidified yet, I think it would be better to hold off on it for now and when we have a stable format and pipeline this would be a good starting point to build from
(Also when you open the pr in the neural-lam repo in the future make sure to use the proper pr template!)

@Prince637-boo
Copy link
Copy Markdown
Author

Thanks @Joltsy10 for the detailed feedback! I completely understand the concern about the timing and the architectural placement.

My goal was to provide a foundation for Gap 6 as early as possible, but I agree that the final graph structure needs to be more stable before we finalize the validation rules. I’m happy to put this on hold for now.

Regarding the location: if the consensus shifts towards hosting the validator in neural-lam, I’m ready to port the logic there once the input formats (Lat/Lon vs Cartesian) are finalized. I'll stay tuned to the discussions in #473 to align with the upcoming standards.

@Prince637-boo Prince637-boo marked this pull request as draft April 5, 2026 16:46
Comment thread src/weather_model_graphs/validator.py Outdated
@Prince637-boo Prince637-boo marked this pull request as ready for review April 5, 2026 18:22
@joeloskarsson
Copy link
Copy Markdown
Contributor

Please use the full PR template for this.

How do you envision this relating to the type of validation in mllam/neural-lam#323 ? It seems a bit double to me to first validate that the graph is correct here, and then validate it again in neural-lam? But maybe this makes sense, if it checks different things?

@Prince637-boo
Copy link
Copy Markdown
Author

Thanks @joeloskarsson for the feedback! I'll update the PR with the proper template immediately.

Regarding the relationship with neural-lam#323, I see them as complementary layers:

1.WMG Validator (Structural/Geometric): This ensures the graph itself is mathematically sound right after construction (e.g., L2 norm consistency on the sphere, bidirectional symmetry between hierarchical levels). It catches 'heavy' geometric errors at the source.

2.Neural-LAM Validator (Runtime/Integrity): This focuses on data loading and ensuring the graph matches the specific model requirements (dimensions, features) just before training.

By validating in WMG, we avoid saving corrupted graph files to disk, which saves time and prevents debugging complex geometry issues during the training phase. I'm happy to refine the scope to ensure there is no overlap!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants