Skip to content

Update Python test workflow to ensure we install dependencies from uv lockfile - #499

Merged
jeancochrane merged 1 commit into
masterfrom
jeancochrane/install-python-deps-from-uv-lockfile
May 20, 2026
Merged

jeancochrane merged 1 commit into
masterfrom
jeancochrane/install-python-deps-from-uv-lockfile

Conversation

@jeancochrane

@jeancochrane jeancochrane commented May 14, 2026 •

Copy link
Copy Markdown
Member

As we found in ccao-data/homeval#158, the way we use uv pip install in the test workflow in this repo will cause uv to ignore python/uv.lock when installing dependencies. Instead, uv will decide which dependency versions to install based on the contents of python/pyproject.toml, but the dependencies in that config file are not locked to specific versions and hashes like they are in python/uv.lock.

This PR updates the test workflow that runs our Python tests to use uv commands that will ensure we use the exact versions of packages pinned in python/uv.lock. We don't need to update the modeling pipeline itself to use different commands because we use the reticulate uv integration to manage Python dependencies for us. This code has a bit more background:

# Use uv to manage Python dependencies for the comps computation.
# This function doesn't actually install the Python dependencies, it just
# configures reticulate to use uv to install those dependencies. Reticulate
# will install the dependencies when we import them. In this particular script,
# we don't import Python dependencies directly, but rather we import the
# comps module which then imports these dependencies.
#
# Because the reticulate uv integration is not very sophisticated, this
# dependency list is duplicated in `python/pyproject.toml`. If you add or
# change any dependencies in this list, make sure to change them there too
reticulate::py_require(
packages = c("numpy==2.2.*", "numba==0.62.*", "pandas==2.3.*"),
python_version = "3.10"
)

Unfortunately, this means that reticulate won't use python/uv.lock either. However, this is the recommended way of using the reticulate uv integration, and py_require doesn't include an option to respect a lockfile, so we would need to roll our own solution to start using the lockfile. I think it's better to use the first-party reticulate uv integration and accept the possibility of different package patch versions than it is to roll our own solution and lock down exact package versions, but I'm open to pushback on that decision.

We don't need a corresponding condo PR for this res PR because the condo PR doesn't have any Python code, so it doesn't use uv.

env:
PYTHONUNBUFFERED: "1"
UV_SYSTEM_PYTHON: 1
UV_FROZEN: 1

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This env var ensures that we don't attempt to update the lockfile every time we run a uv command in this workflow. That wouldn't be a big deal, but it would be a waste of time since we wouldn't ever use the new lockfile even if the workflow were to generate one. See the docs for Automatic locking and syncing for more background.

working-directory: python
shell: bash
run: uv pip install .[tests]
run: uv sync --extra tests

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is equivalent to uv pip install ., but it respects the contents of the lockfile (in this case, python/uv.lock).

shell: bash
working-directory: python
run: pytest
run: uv run pytest

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wrapping this pytest call in uv run tells uv to load the virtual environment in the current working directory (in this case, python/) before running the command.

@jeancochrane
jeancochrane marked this pull request as ready for review May 14, 2026 21:25

@wrridgeway wrridgeway left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for documenting this so well.

@jeancochrane
jeancochrane merged commit 1fd4fcb into master May 20, 2026
6 checks passed
@jeancochrane
jeancochrane deleted the jeancochrane/install-python-deps-from-uv-lockfile branch May 20, 2026 15:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants