Fix missing PEFT availability check when passing peft_config to experimental trainers

[albertvillanova]

Fix missing PEFT availability check when passing peft_config to experimental trainers.

This PR adds improved error handling for cases where a peft_config is provided without the required peft library installed in both the distillation and TPO experimental trainers. This ensures users receive a clear, actionable error message if they attempt to use PEFT features without having the necessary dependencies.

Related to:

• Fix missing PEFT validation when passing peft_config to core trainers #5664

Changes

Error handling improvements for PEFT:

• Added a check in trl/experimental/distillation/distillation_trainer.py to raise an ImportError with installation instructions if peft_config is provided but the peft library is missing.
• Added a similar check in trl/experimental/tpo/tpo_trainer.py to ensure the peft library is installed when peft_config is used, raising an informative ImportError if not.

---

Note

Low Risk
Changes are limited to trainer initialization validation and error messaging around PEFT, with minimal impact beyond earlier/clearer failures for misconfigured setups.

Overview
Standardizes PEFT setup across experimental trainers by validating peft_config early and failing fast with clearer errors.

When peft_config is provided, trainers now (1) raise an actionable ImportError if peft isn’t installed (with pip install trl[peft] guidance) and (2) enforce that peft_config is a peft.PeftConfig instance (e.g. LoraConfig), before proceeding; several trainers also update PEFT imports to include PeftConfig and keep the existing guard against passing an already-wrapped PeftModel alongside a new config.

^{Reviewed by Cursor Bugbot for commit 70b898f. Bugbot is set up for automated code reviews on this repo. Configure here.}

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[qgallouedec]

same review as #5664

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit d236c55. Configure here.}