Fix peft_config type hint in experimental trainers

[albertvillanova]

Fix peft_config type hint in experimental trainers.

This PR refactors the PEFT configuration handling in experimental trainer classes (BCOTrainer, CPOTrainer, KTOTrainer, and ORPOTrainer). The main improvement is replacing the previous use of a generic dict for peft_config with the more specific and type-safe PeftConfig from the peft library. This change enhances type safety, improves documentation clarity, and ensures consistency across the codebase.

Changes

PEFT configuration improvements:

• The peft_config parameter in all experimental trainer classes is now typed as PeftConfig | None instead of dict | None, and the corresponding docstrings have been updated to reflect this change. This ensures better type checking and clearer intent for users and developers.

• The import statements in all trainer files have been updated to import PeftConfig from the peft library, alongside the existing PEFT-related imports.

---

Note

Low Risk
Low risk: changes are limited to type hints/imports and docstrings around peft_config, with no functional training logic modifications.

Overview
Type-safety cleanup for PEFT configs in experimental trainers.

Updates BCOTrainer, CPOTrainer, KTOTrainer, ORPOTrainer, PRMTrainer, NashMDTrainer, and XPOTrainer to accept peft_config as PeftConfig | None (and documents it as such) instead of dict | None, and adds the corresponding PeftConfig imports where PEFT is available.

^{Reviewed by Cursor Bugbot for commit a719702. Bugbot is set up for automated code reviews on this repo. Configure here.}

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 1ea5f27. Configure here.}

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.