Skip to content

Fix spurious KL gradients for zero-std reward groups in GRPOTrainer #5640

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
robrui wants to merge 2 commits into from

Fix paper link: use huggingface.co/papers/ instead of arxiv.org/abs/ …

cfc4e54
Select commit
Loading
Failed to load commit list.
Closed

Fix spurious KL gradients for zero-std reward groups in GRPOTrainer #5640

Fix paper link: use huggingface.co/papers/ instead of arxiv.org/abs/ …
cfc4e54
Select commit
Loading
Failed to load commit list.
Cursor / Cursor Bugbot completed Apr 26, 2026 in 8m 12s

Bugbot Review

Bugbot Analysis Progress (8m 15s elapsed)

✅ Gathered PR context (2s)
✅ Completed bug detection (8m 8s)
✅ Posted analysis results (5s)

Final Result: Bugbot completed review and found 1 potential issue.

Request ID: serverGenReqId_a48fb750-8c1e-4436-b2e1-658d7526562f

Details

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit cfc4e54. Configure here.

View more details on Cursor