Relax deps, transformers 5.x compat, comet as optional extra

[smirnovlad]

Summary

• Remove spacy<3.8.0 upper bound → spacy>=3.4.0

  • Allows newer thinc/numpy versions needed for vLLM compatibility

• Remove transformers upper bound → transformers>=4.50.0

  • Added forward-compatibility code for transformers 5.x breaking changes:
    • BeamScorer removal → None fallback
    • Output class renames → try/except imports
    • batch_encode_plus removal → replaced with tokenizer()
    • AutoModelForVision2Seq rename → fallback to AutoModelForImageTextToText
  • Added _SanitizeLogitsProcessor to handle inf/nan logits edge cases

• Move unbabel-comet to optional extras → pip install lm-polygraph[comet]

  • unbabel-comet pins numpy<2.0 which conflicts with vLLM (numpy>=2.0)
  • Comet import guarded with try/except ImportError in __init__.py
  • Lazy evaluate import in comet.py

Install paths

Use case	Command	numpy	transformers
Full (with comet)	pip install lm-polygraph[comet]	<2.0	<5.0
Without comet (for vLLM)	pip install lm-polygraph	any	any
With comet + numpy 2.x	pip install lm-polygraph then pip install unbabel-comet --no-deps	2.x	any

Test plan

• CI passes (single job, no transformers version matrix)
• from lm_polygraph.generation_metrics import Comet returns None when comet not installed
• pip install lm-polygraph[comet] installs comet correctly
• Existing tests pass with both transformers 4.x and 5.x

[smirnovlad]

Note on _SanitizeLogitsProcessor and numpy 2.x

Why relax spacy<3.8.0?

vLLM (and other modern ML packages) requires numpy>=2.0. But:

• spacy<3.8.0 → installs thinc 8.2.x → compiled against numpy 1.x
• numpy 1.x and numpy 2.x are binary incompatible (ValueError: numpy.dtype size changed)

So lm-polygraph with spacy<3.8.0 is uninstallable alongside vLLM. Relaxing to spacy>=3.4.0 allows spacy 3.8+ → thinc 8.3+ which is built for numpy 2.x.

Why the logits sanitizer?

The numpy 1.x → 2.x change subtly affects floating point behavior in PyTorch CPU operations. Specifically, bloomz-560m running on CPU with numpy 2.x occasionally produces inf/nan logits during generation, which crashes torch.multinomial in the sampling path:

RuntimeError: probability tensor contains either `inf`, `nan` or element < 0

The same model + code + seed works fine with numpy 1.x (the main branch) — the numerical difference is purely from the numpy version.

The _SanitizeLogitsProcessor added in this PR runs first in the logits processor chain and clamps any inf/nan values to finite numbers via torch.nan_to_num(). This is a defensive fix that makes the code robust regardless of the numpy version or hardware.

[smirnovlad]

Root cause of slow CI tests

Found the issue. When we made unbabel-comet optional (commented out in requirements.txt), we removed an implicit transformers<5 pin — comet was the dependency that constrained transformers to 4.x.

Without that constraint, pip install . now resolves to transformers 5.x even for the "default" CI job.

In transformers 5.x, generation parameters like temperature, top_k, top_p must be passed via a GenerationConfig object — passing them as loose kwargs to model.generate() results in:

The following generation flags are not valid and may be ignored: ['temperature']

This meant temperature=0.7 from test configs was silently ignored → model used default temperature=1.0 → generated much longer outputs → NLI calculator had to process far more tokens (310s/batch instead of ~30s) → CI tests took 25+ minutes instead of ~7 minutes → runner killed with SIGTERM.

Fix: wrap generation params in GenerationConfig before calling model.generate(). This is backwards-compatible with transformers 4.x too.

[smirnovlad]

Summary

Goal: Make lm-polygraph installable without any post-install patches (sed hacks), so we can release ThinkBooster as a standalone PyPI package. Currently ThinkBooster's setup.sh patches lm-polygraph's requirements.txt after cloning — this won't work for a PyPI release where users just do pip install thinkbooster.

What we did:

1. Removed spacy<3.8.0 upper bound — the old pin forced thinc 8.2.x → numpy 1.x, which conflicts with vLLM (numpy>=2.0)

2. Removed transformers upper bound (was <4.52.0) — added forward-compatibility code for transformers 5.x (try/except imports for removed/renamed classes, batch_encode_plus replacement, logits sanitizer for inf/nan edge cases)

3. Moved unbabel-comet to optional [comet] extra — unbabel-comet pins numpy<2.0, and there's no version of comet that supports numpy 2.x. Since vLLM requires numpy>=2.0, they can't coexist. By making comet optional:

   • pip install lm-polygraph — works with vLLM/numpy 2.x (what ThinkBooster needs)
   • pip install lm-polygraph[comet] — full install with comet for standalone usage

Result: After this PR is merged, ThinkBooster can simply list lm-polygraph as a dependency in pyproject.toml — no patches, no sed, clean PyPI-compatible install.