Migrate dependency management to pyproject.toml with uv

[matrixise]

Description

Complete migration from the legacy pip-tools workflow using requirements/*.in files to a modern Python packaging approach using pyproject.toml with uv for dependency management.

This migration consolidates all dependency declarations into a single source of truth (pyproject.toml) following PEP 621 (project metadata) and PEP 735 (dependency groups) standards.

Type of Change

• Refactoring
• Documentation update
• Bug fix
• New feature

Key Changes

Structure

• Replaced requirements/main.in, requirements/dev.in, requirements/production.in → pyproject.toml
• Generated uv.lock as universal lock file (102 packages resolved)
• Auto-generate requirements.txt for Heroku deployment compatibility
• Centralized all tool configurations (ruff, coverage, isort) in pyproject.toml
• Using Hatchling as build backend

New Workflow

# Update dependencies
uv lock

# Install dependencies (development)
uv sync

# Install dependencies (production)
uv sync --no-dev --group production

# Export for Heroku
task dependencies:export

# Upgrade specific package
task dependencies:upgrade:package PACKAGE=django

Files Modified

• pyproject.toml - New single source of truth
• uv.lock - Universal lock file
• Taskfile.yaml - Updated dependency management tasks
• toast.yml - Updated Docker-based dependency tasks
• Dockerfile - Use uv sync instead of pip install
• .github/workflows/test.yml - Updated CI to use uv
• CLAUDE.md, README.md, DEVELOPMENT.md, CONTRIBUTING.md - Updated documentation

Files Removed

• requirements/main.in, requirements/dev.in, requirements/production.in
• requirements/main.txt, requirements/dev.txt, requirements/production.txt
• requirements/ directory (now empty)
• pythonie/setup.py (replaced by pyproject.toml)

Benefits

1. Single source of truth - All dependencies in pyproject.toml
2. Faster resolution - uv is 10-100x faster than pip
3. Modern standards - PEP 621 + PEP 735 (approved Oct 2024)
4. Centralized config - All tool settings in one file
5. Better reproducibility - Universal lock file with hashes
6. Simplified commands - One command to install instead of three

How to Test

Local Setup

# Clean environment
rm -rf .venv pythonie-venv

# Install with new system
uv sync

# Run tests
python pythonie/manage.py test pythonie --settings=pythonie.settings.tests --verbosity=2

Docker Setup

# Rebuild Docker image
task docker:build

# Run tests in Docker
task tests

Expected Results

• ✅ All dependencies installed successfully
• ✅ All 33 tests pass
• ✅ requirements.txt generated with correct header
• ✅ Development workflow unchanged from user perspective

Testing Performed

• ✅ Clean installation with `uv sync`
• ✅ All 33 tests pass (ran in 0.156s)
• ✅ Generated `requirements.txt` for Heroku with auto-generated header
• ✅ Verified all dependencies resolved correctly (102 packages)

Checklist

• Code follows project style guidelines
• Tests pass locally (`uv sync` + full test suite)
• New tests added (N/A - infrastructure change)
• Documentation updated (CLAUDE.md, README.md, DEVELOPMENT.md, CONTRIBUTING.md)
• Backward compatibility maintained (requirements.txt still generated for Heroku)
• All old dependency files removed
• uv.lock committed to version control

Migration Impact

Developers

• Must run `uv sync` instead of `pip install -r requirements.txt`
• New commands via Taskfile (but similar to before)

CI/CD

• GitHub Actions updated to use `uv sync`
• All tests passing with new setup

Deployment (Heroku)

• No impact - still uses `requirements.txt` (auto-generated)
• Deployment process unchanged

References

• Fixes 🔧 Migrate from pip to uv for dependency management #171
• PEP 621: Storing project metadata in pyproject.toml
• PEP 735: Dependency Groups in pyproject.toml (Oct 2024)
• uv documentation

---

Note: This is a significant infrastructure change, but maintains full backward compatibility for deployment while modernizing the development workflow.

[ulgens]

Would you consider setup-uv action instead of setup-python? It can install Python versions just fine, and yields a cleaner result in overall config in my experience.

[ulgens]

Are these exports still necessary? I tried to check how they are being used after uv migration but I couldn't catch the usecase.

[matrixise]

Hi @ulgens,
I haven't checked if Heroku supports uv yet, but the main idea was to keep full compatibility with Heroku and the requirements files. This ensures the deployment pipeline continues to work as expected without any changes.

[ulgens]

Got it, thank you.

Checking Heroku docs now and it seems they already have the uv support in place: https://devcenter.heroku.com/changelog-items/3238 I also checked

• Which version they support: https://github.com/heroku/buildpacks-python/blob/main/requirements/uv.txt 0.9.18 at time of this comment, a release from 3 weeks ago, not bad.
• Do they support a custom version definition, but couldn't find anything about it.