Skip to content

Conversation

@coreyjadams
Copy link
Collaborator

@coreyjadams coreyjadams commented Dec 3, 2025

This PR is about dependencies and installation. Goals:

  • Ensure uv is first-class
  • Ensure pip install works
  • Ensure default deps are broadly available and won't hold up other pieces.

To this end, the core deps are listed, and there is linting against them. We can't import non-core deps "bare" any more.

Further, optional deps are almost* entirely protected with check_version_spec, etc., to raise runtime errors at first use only. Meaning, a missing dep will not break the rest of the code, and won't even crash until a user attempts to use it. And I've tried to list out the optional deps per package, too, though some could use versioning.

Everything below is outdated. I left it to show the original design.

Ensure that the pip dependency group exclusively matches the resolution of the dependency-groups from Pep 735 that uv enables.

Requirements are to ensure pip install . still works, with or without uv, even though we want uv to be the sole source of truth. Rather than maintaining two dep lists, especially since one is nested, I made a pre-commit step to ensure that the pip requirements is always exactly what dependency-groups specifies.

Basically, the workflow is this:

  • pre-commit will run test/ci_tests/sync_deps.py
  • sync_deps.py ingests the pyproject.yaml file.
    • It recurses through the physicsnemo dependency group (no extras) and resolves all the groups and dependencies.
    • It generates a flat list of dependencies.
    • De-duplication is automatic, and if the version strings don't agree it's an error.
  • Then:
    • If the list agrees, PASS
    • If the list disagrees, it will overwrite the dependency section of pyproject.toml and fail pre-commit.

This should achieve the following:

  • if dependency-group dependencies update, we will be unable to commit that code to the repo unless the pip install . dependencies also update.
  • If the pip dependencies update, that update will be destroyed automatically.

PhysicsNeMo Pull Request

Description

Checklist

  • I am familiar with the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.
  • The CHANGELOG.md is up to date with these changes.
  • An issue is linked to this pull request.

Dependencies

Review Process

All PRs are reviewed by the PhysicsNeMo team before merging.

Depending on which files are changed, GitHub may automatically assign a maintainer for review.

We are also testing AI-based code review tools (e.g., Greptile), which may add automated comments with a confidence score.
This score reflects the AI’s assessment of merge readiness and is not a qualitative judgment of your work, nor is
it an indication that the PR will be accepted / rejected.

AI-generated feedback should be reviewed critically for usefulness.
You are not required to respond to every AI comment, but they are intended to help both authors and reviewers.
Please react to Greptile comments with 👍 or 👎 to provide feedback on their accuracy.

resolution of the dependency-groups from Pep 735 that uv enables.
@greptile-apps
Copy link
Contributor

greptile-apps bot commented Dec 3, 2025

Greptile Overview

Greptile Summary

This PR introduces a pre-commit hook and script to automatically synchronize project.dependencies with the dependency-groups defined in pyproject.toml, ensuring PEP 735 compliance with uv's dependency resolution.

  • Added sync_deps.py script that resolves the utils dependency group using DFS traversal, detects cycles, validates version consistency, and auto-generates the dependencies list
  • Added pre-commit hook to run this script on every commit, failing if dependencies need updating
  • Fixed a bug in the physicsnemo group that referenced domain-parallel (hyphen) instead of the actual group domain_parallel (underscore)
  • Restructured dependency hierarchy: models now includes domain_parallel instead of nn directly, and physicsnemo includes models instead of the broken domain-parallel reference

Important Files Changed

File Analysis

Filename Score Overview
test/ci_tests/sync_deps.py 4/5 New script that auto-generates project.dependencies from dependency-groups, with DFS resolution and cycle detection. Shebang placement is non-standard but functional.
pyproject.toml 5/5 Fixed broken domain-parallel reference (was missing underscore), added auto-generated dependencies section, and restructured dependency group hierarchy.
.pre-commit-config.yaml 5/5 Added new sync-deps pre-commit hook that runs sync_deps.py on every commit.

Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

3 files reviewed, 1 comment

Edit Code Review Agent Settings | Greptile

# limitations under the License.


#!/usr/bin/env python3
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

style: Shebang line should be at line 1 to work when executing the script directly. Currently it's after the license header, which means ./sync_deps.py won't work, only python sync_deps.py. This is fine since the pre-commit hook calls it via python, but worth noting for consistency.

@coreyjadams coreyjadams changed the title Ensure that the pip dependency group exclusively matches the UV <---> Pip must stay in sync. Dec 3, 2025
Now, we allow unbare external imports anywhere.  But the import list is smaller,
and more easily managed with uv / pip equally.  extras are now
chained and built in the optional deps.
"zarr>=2.14.2",
"jaxtyping",
{include-group="nn"},
"nvidia-physicsnemo[utils-extras]",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No natten? 😅

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should add it! But unless it's easy to install cross-platform, it needs to go into an optional dep.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nn-extras here is optional right? Natten should def be optional regardless, but I think nn-extras is the right place for it

healpix = [
"earth2grid",
]

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems the gnns and healpix here are more just general deps groups associated with certain functionality/use-cases but not a part of the base pnm subpackages (nn, models, datapipes ...). Maybe separate them out with a small comment header for clarity? Like

# Use-case Specific Dependency Groups

or something...


# Earth2 grid is not on pypi ....
[tool.uv.sources]
earth2grid = { url = "https://github.com/NVlabs/earth2grid/archive/main.tar.gz" }
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

May want to look at the earth2studio toml to see the various things Nick added to handle earth2grid...I think there is also a need to include --no-build-isolation

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Relevant lines appear to be:

no-build-isolation-package = ["earth2grid", "flash-attn", "torch-harmonics"]
[tool.uv.extra-build-dependencies]
earth2grid = ["setuptools", "torch"]

[tool.uv.sources]
...
earth2grid = { git = "https://github.com/NVlabs/earth2grid.git", rev = "661445e2c68edc76f52632aa0528af482357f1b8" }

I can add the missing pieces, though I suspect we don't need to lock to a revision.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I need to double check with Nick when he's back why he pinned version, it may have just been out of "abundance of caution" :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants