Skip to content

Relax dependency bounds and make comet optional#448

Closed
smirnovlad wants to merge 3 commits intomainfrom
fix/relax-deps-only
Closed

Relax dependency bounds and make comet optional#448
smirnovlad wants to merge 3 commits intomainfrom
fix/relax-deps-only

Conversation

@smirnovlad
Copy link
Copy Markdown
Collaborator

Summary

  • Remove spacy<3.8.0 upper bound: allows pip to resolve newer thinc / numpy>=2.0, which is required for compatibility with vLLM and other packages that need numpy 2.x
  • Pin transformers>=4.50.0,<5: transformers 5.x introduces breaking API changes (removed BeamScorer, renamed output classes, removed batch_encode_plus). This pin keeps the current code working while we prepare a separate forward-compatibility PR.
  • Make unbabel-comet optional: commented out from requirements.txt, lazy-imported in comet.py, with try/except in __init__.py. This removes the hard dependency that was pulling in conflicting transformers version constraints.
  • Updated README with an "Optional dependencies" section explaining how to install unbabel-comet separately.

Motivation

This is a prerequisite for publishing ThinkBooster as a PyPI package — lm-polygraph must be installable without manual patching of dependency pins.

What this PR does NOT include

Transformers 5.x forward-compatibility code changes (try/except import fallbacks, batch_encode_plus replacements, logits sanitizer). Those will come in a separate PR.

Test plan

  • CI passes with existing tests (no code changes, only dependency resolution)
  • pip install in a clean environment resolves without conflicts
  • COMET metric works when unbabel-comet is installed separately
  • COMET-dependent code gracefully skips when unbabel-comet is not installed

- Remove spacy<3.8.0 upper bound (allows numpy 2.x via newer thinc)
- Pin transformers<5 to avoid breaking API changes
- Make unbabel-comet optional (lazy import in comet.py)
- Add optional dependencies section to README
@smirnovlad
Copy link
Copy Markdown
Collaborator Author

unbabel-comet pins numpy<2.0.0,>=1.20.0 — this conflicts with vLLM which requires numpy>=2.0. That's the reason we strip comet in ThinkBooster's setup.sh (sed -i '/unbabel-comet/d').

The transformers<5 constraint from comet is indeed fine since we already pin that ourselves. But the numpy pin is the actual blocker.

Making comet optional here lets lm-polygraph install cleanly alongside vLLM without needing post-install sed hacks.

Instead of commenting out comet entirely, move it to
pyproject.toml optional-dependencies so users can install
via `pip install lm-polygraph[comet]`.

This avoids the numpy<2.0 conflict with vLLM while keeping
comet easily installable for users who need it.
@smirnovlad
Copy link
Copy Markdown
Collaborator Author

Superseded by #446 which now includes both dependency changes and transformers 5.x compat code.

@smirnovlad smirnovlad closed this Apr 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant