Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for PPVCtrl #10625

Open
2 tasks done
owlowlohh opened this issue Jan 22, 2025 · 0 comments
Open
2 tasks done

Add support for PPVCtrl #10625

owlowlohh opened this issue Jan 22, 2025 · 0 comments

Comments

@owlowlohh
Copy link

owlowlohh commented Jan 22, 2025

Model/Pipeline/Scheduler description

I recently came across an impressive work called PPVCtrl, a controllable video generation model. It leverages an auxiliary condition encoder to transform a text-to-video generation model into a customizable video generator, all without retraining the original generator. It's akin to ControlNet, but for video generation.

Prompt Reference Image Control Videos PP-VCtrl-5B-T2V PP-VCtrl-5B-I2V
Group of fishes swimming in aquarium. Reference Conrotl Videos PP-VCtrl T2V PP-VCtrl I2V
A boat with a flag on it is sailing on the sea. Reference Control Videos PP-VCtrl T2v PP-VCtrl I2v
Prompt Reference Image Control Videos PP-VCtrl-5B-T2V PP-VCtrl-5B-I2V
A rider in a dark helmet and white breeches is atop a chestnut horse... Reference Conrotl Videos PP-VCtrl T2V PP-VCtrl I2V
A dark gray Mini Cooper is parked on a city street... Reference Control Videos PP-VCtrl T2v PP-VCtrl I2v
Prompt Reference Image Pose Videos PP-VCtrl-5B-I2V
A young man with curly hair and a red t-shirt featuring a white logo is seen in various states of motion... Reference 1 Pose Videos PP-VCtrl 1
A woman models an Adrianna Papell women's gown, featuring a sleeveless... Reference 2 Pose Videos PP-VCtrl 2

Open source status

  • The model implementation is available.
  • The model weights are available (Only relevant if addition is not a scheduler).

Provide useful links for the implementation

https://github.com/PaddlePaddle/PaddleMIX/tree/develop/ppdiffusers/examples/ppvctrl

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant