Skip to content

Conversation

@yuanwu2017
Copy link
Contributor

What does this PR do?

Fixes # (issue)

Before submitting

  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • Did you make sure to update the documentation with your changes?
  • Did you write any new necessary tests?

Signed-off-by: yuanwu <yuan.wu@intel.com>
Signed-off-by: yuanwu <yuan.wu@intel.com>
Signed-off-by: yuanwu <yuan.wu@intel.com>
@yuanwu2017 yuanwu2017 requested a review from regisss as a code owner December 26, 2024 06:42
Signed-off-by: yuanwu <yuan.wu@intel.com>
@yuanwu2017 yuanwu2017 marked this pull request as draft January 12, 2025 22:20
Signed-off-by: yuanwu <yuan.wu@intel.com>
Signed-off-by: yuanwu <yuan.wu@intel.com>
Signed-off-by: yuanwu <yuan.wu@intel.com>
Signed-off-by: yuanwu <yuan.wu@intel.com>
Signed-off-by: yuanwu <yuan.wu@intel.com>
Signed-off-by: yuanwu <yuan.wu@intel.com>
Signed-off-by: yuanwu <yuan.wu@intel.com>
Signed-off-by: yuanwu <yuan.wu@intel.com>
Signed-off-by: yuanwu <yuan.wu@intel.com>
@yuanwu2017
Copy link
Contributor Author

yuanwu2017 commented Jan 19, 2025

Command:
BF16
PT_HPU_MAX_COMPOUND_OP_SIZE=1 python image_to_video_generation.py --model_name_or_path "ali-vilab/i2vgen-xl" --image_path "https://huggingface.co/datasets/diffusers/docs-images/resolve/main/i2vgen_xl_images/img_0009.png" --num_videos_per_prompt 1 --video_save_dir ./i2vgen_xl --num_inference_steps 50 --use_habana --use_hpu_graphs --gaudi_config Habana/stable-diffusion --gif --num_frames 16 --prompts "Papers were floating in the air on a table in the libra ry" --negative_prompts "Distorted, discontinuous, Ugly, blurry, low resolution, motionless, static, disfigured, disconnected limbs, Ugly faces, incomplete arms" --seed 8888 --sdp_on_bf16 --bf16

Result:
image

FP32
PT_HPU_MAX_COMPOUND_OP_SIZE=1 python image_to_video_generation.py --model_name_or_path "ali-vilab/i2vgen-xl" --image_path "https://huggingface.co/datasets/diffusers/docs-images/resolve/main/i2vgen_xl_images/img_0009.png" --num_videos_per_prompt 1 --video_save_dir ./i2vgen_xl --num_inference_steps 50 --use_habana --use_hpu_graphs --gaudi_config Habana/stable-diffusion --gif --num_frames 16 --prompts "Papers were floating in the air on a table in the libra ry" --negative_prompts "Distorted, discontinuous, Ugly, blurry, low resolution, motionless, static, disfigured, disconnected limbs, Ugly faces, incomplete arms" --seed 8888

Result:
image

Because the file size limitation, I uploaded a video of num_frames=10.

gen_video_00

Signed-off-by: yuanwu <yuan.wu@intel.com>
@yuanwu2017 yuanwu2017 changed the title I2v debug Enable the i2vgen pipeline Jan 19, 2025
@yuanwu2017 yuanwu2017 marked this pull request as ready for review January 19, 2025 16:03
Signed-off-by: yuanwu <yuan.wu@intel.com>
Signed-off-by: yuanwu <yuan.wu@intel.com>
Signed-off-by: yuanwu <yuan.wu@intel.com>
Signed-off-by: yuanwu <yuan.wu@intel.com>
Signed-off-by: yuanwu <yuan.wu@intel.com>
@yuanwu2017
Copy link
Contributor Author

Performance:
Gaudi2:
image
throughput=0.006999005392688631 sample/sec
A100:
image
throughtput=0.0025254897026770477 sample/sec

@yuanwu2017
Copy link
Contributor Author

@libinta @regisss Please help to review.

Signed-off-by: yuanwu <yuan.wu@intel.com>
yuanwu2017 and others added 5 commits January 30, 2025 22:59
Co-authored-by: Ilyas Moutawwakil <57442720+IlyasMoutawwakil@users.noreply.github.com>
Signed-off-by: yuanwu <yuan.wu@intel.com>
Signed-off-by: yuanwu <yuan.wu@intel.com>
Signed-off-by: yuanwu <yuan.wu@intel.com>
@yuanwu2017
Copy link
Contributor Author

yuanwu2017 commented Jan 31, 2025

pass ci tests
image

Signed-off-by: yuanwu <yuan.wu@intel.com>
@yuanwu2017
Copy link
Contributor Author

@regisss @IlyasMoutawwakil @libinta Please review.

@HuggingFaceDocBuilderDev

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Signed-off-by: yuanwu <yuan.wu@intel.com>
Signed-off-by: yuanwu <yuan.wu@intel.com>
Signed-off-by: yuanwu <yuan.wu@intel.com>
@yuanwu2017
Copy link
Contributor Author

As @libinta mentioned, they are simplifying the README, Remove the commands in README.

@libinta libinta added the run-test Run CI for PRs from external contributors label Feb 6, 2025
Signed-off-by: yuanwu <yuan.wu@intel.com>
Copy link
Collaborator

@regisss regisss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we also add this new pipeline to the table in the README and doc?

yuanwu2017 and others added 4 commits February 7, 2025 21:45
@yuanwu2017
Copy link
Contributor Author

Should we also add this new pipeline to the table in the README and doc?

Done

Copy link
Collaborator

@regisss regisss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@regisss regisss merged commit 18449ba into huggingface:main Feb 7, 2025
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

run-test Run CI for PRs from external contributors

Projects

None yet

Development

Successfully merging this pull request may close these issues.

None yet

6 participants