Skip to content

Conversation

@xingyaoww
Copy link
Contributor

This PR adds a test for the ref_input parameter that was introduced in #467.

Changes

  • Add test_ref_input.py to verify the ref_input parameter works correctly in LigerFusedLinearPreferenceBase
  • Test ensures that:
    • Policy model outputs (chosen_logps, rejected_logps) are identical when using ref_input vs not using it
    • Final loss and aux outputs differ when using ref_input vs using input_chunk for reference model
  • Uses same parametrization as other tests for consistency

Testing

The test verifies that:

  1. When ref_input is not provided, the reference model uses input_chunk
  2. When ref_input is provided, the reference model uses it instead of input_chunk
  3. The policy model outputs remain unchanged regardless of ref_input
  4. The final loss differs when using different inputs for the reference model

Follows up on #467 which added ref_input parameter support.

openhands-agent and others added 3 commits December 11, 2024 21:37
- Add test to verify ref_input parameter works correctly in LigerFusedLinearPreferenceBase
- Test ensures policy outputs are identical but losses differ when using ref_input
- Follows PR linkedin#467 which added ref_input parameter support
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants