Skip to content

Add ability to give custom rollout log function | Custom log hook where users can define how they want to log samples. #6007

@yeshsurya

Description

@yeshsurya

Feature request

  • New config: trainer.custom_rollout_log_function — a dotted Python path
  • Hook receives keyword args: step, inputs, outputs, scores, ground_truths, reward_extra_infos, batch (raw DataProto), timing, config
  • Return True → skip default logging; False → default continues
  • Called for both train and eval rollouts
  • Zero overhead when not configured
  • Includes an example showing wandb Table logging of per-sample prompt/response/reward
  • Similar to slime's --custom-rollout-log-function-path

Motivation

So that I can use different metrics providers like wandb etc to see how my rollouts are looking

Your contribution

I'll try to contribute this feature.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions