Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix the incorrect output of sdxl inpaint #1737

Merged
merged 3 commits into from
Feb 5, 2025
Merged

Conversation

yuanwu2017
Copy link
Contributor

@yuanwu2017 yuanwu2017 commented Jan 31, 2025

What does this PR do?

Fix the incorrect output of sdxl inpaint.
Command:
python text_to_image_generation.py --model_name_or_path diffusers/stable-diffusion-xl-1.0-inpainting-0.1 --base_image https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/inpaint.png --mask_image https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/inpaint_mask.png --prompts "concept art digital painting of an elven castle, inspired by lord of the rings, highly detailed, 8k" --seed 42 --scheduler euler_discrete --num_images_per_prompt 12 --batch_size 4 --image_save_dir ./xl2 --use_habana --use_hpu_graphs --gaudi_config Habana/stable-diffusion --sdp_on_bf16 --bf16

Before:

image Images(only attach first four images ):

image_1

image_2

image_3

image_4

After:
image
Images(only attach first four images ):
image_1
image_2
image_3
image_4

Seeing from the results, the performance has no decline. The output pictures are more fine

Fixes # (issue)

Before submitting

  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • Did you make sure to update the documentation with your changes?
  • Did you write any new necessary tests?

@yuanwu2017 yuanwu2017 requested a review from regisss as a code owner January 31, 2025 13:24
@yuanwu2017
Copy link
Contributor Author

@regisss @dsocek Please help to review.

@yuanwu2017
Copy link
Contributor Author

pass ci test.
image

@libinta libinta added the run-test Run CI for PRs from external contributors label Feb 3, 2025
@dsocek
Copy link
Contributor

dsocek commented Feb 4, 2025

@yuanwu2017 did you test other schedulers? (e.g. default)? Also, did you try different seeds, prompts to really see that scheduler visually impairs images if we call set_begin_index vs. if we do not

Copy link
Contributor

@dsocek dsocek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please run more tests and show proof that we can safely remove this. For default scheduler removing this may cause issues. Also, can we do a larger sweep of prompts/seeds to justify removal

@dsocek
Copy link
Contributor

dsocek commented Feb 4, 2025

@regisss Checked old jira info. We were getting error when using init_step_index without setting set_begin_index first (for ddim scheduler). This was around SW stack 1.16/1.17 and SDXL. I just did some tests on my side with 1.19 and seems this issue is not present anymore. @yuanwu2017 can you also test this? If this is the case, we can remove the set_begin_index if it improves visual quality in inpainting

@dsocek
Copy link
Contributor

dsocek commented Feb 4, 2025

Update: @yuanwu2017 I tested 1.19 with all schedulers and the issue we saw in older releases is not showing. I also did observe better details in generated images, so I think it is safe to remove set_begin_index code. @yuanwu2017 could you also remove from other 3 SDXL pipelines in this PR? Thanks!

@yuanwu2017
Copy link
Contributor Author

Update: @yuanwu2017 I tested 1.19 with all schedulers and the issue we saw in older releases is not showing. I also did observe better details in generated images, so I think it is safe to remove set_begin_index code. @yuanwu2017 could you also remove from other 3 SDXL pipelines in this PR? Thanks!

Ok. I will remove. Thanks.

@HuggingFaceDocBuilderDev

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Copy link
Collaborator

@regisss regisss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@regisss regisss merged commit 997f04d into huggingface:main Feb 5, 2025
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
run-test Run CI for PRs from external contributors
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants