You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I recently came across an impressive work called PPVCtrl, a controllable video generation model. It leverages an auxiliary condition encoder to transform a text-to-video generation model into a customizable video generator, all without retraining the original generator. It's akin to ControlNet, but for video generation.
Prompt
Reference Image
Control Videos
PP-VCtrl-5B-T2V
PP-VCtrl-5B-I2V
Group of fishes swimming in aquarium.
A boat with a flag on it is sailing on the sea.
Prompt
Reference Image
Control Videos
PP-VCtrl-5B-T2V
PP-VCtrl-5B-I2V
A rider in a dark helmet and white breeches is atop a chestnut horse...
A dark gray Mini Cooper is parked on a city street...
Prompt
Reference Image
Pose Videos
PP-VCtrl-5B-I2V
A young man with curly hair and a red t-shirt featuring a white logo is seen in various states of motion...
A woman models an Adrianna Papell women's gown, featuring a sleeveless...
Open source status
The model implementation is available.
The model weights are available (Only relevant if addition is not a scheduler).
Model/Pipeline/Scheduler description
I recently came across an impressive work called PPVCtrl, a controllable video generation model. It leverages an auxiliary condition encoder to transform a text-to-video generation model into a customizable video generator, all without retraining the original generator. It's akin to ControlNet, but for video generation.
Open source status
Provide useful links for the implementation
https://github.com/PaddlePaddle/PaddleMIX/tree/develop/ppdiffusers/examples/ppvctrl
The text was updated successfully, but these errors were encountered: