PPO on the pi-0 model

[prahalad12345]

The repo has been a massive help!! I want to integrate PPO to optimize the pi-0 bridge.pt policy , based on a specific requirement. It would be great if you could provide research or codebases that could assist with the implementation of the same. I attempted to modify the infer_action , but concerns regarding the generation of logprob and creation of an evaluation function for the PPO ?

[allenzren]

Hey my other work is quite relevant actually https://github.com/irom-princeton/dppo. It would be awesome if you can apply it to fine-tune the action expert.

[prahalad12345]

Thanks . Will look into it and reach back to you if I face some trouble regarding the same