-
Notifications
You must be signed in to change notification settings - Fork 227
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix the incorrect output of sdxl inpaint #1737
Conversation
optimum/habana/diffusers/pipelines/stable_diffusion_xl/pipeline_stable_diffusion_xl_inpaint.py
Show resolved
Hide resolved
@yuanwu2017 did you test other schedulers? (e.g. default)? Also, did you try different seeds, prompts to really see that scheduler visually impairs images if we call |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please run more tests and show proof that we can safely remove this. For default scheduler removing this may cause issues. Also, can we do a larger sweep of prompts/seeds to justify removal
@regisss Checked old jira info. We were getting error when using |
Update: @yuanwu2017 I tested 1.19 with all schedulers and the issue we saw in older releases is not showing. I also did observe better details in generated images, so I think it is safe to remove |
Ok. I will remove. Thanks. |
Signed-off-by: yuanwu <[email protected]>
Signed-off-by: yuanwu <[email protected]>
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
Signed-off-by: yuanwu <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM!
What does this PR do?
Fix the incorrect output of sdxl inpaint.
Command:
python text_to_image_generation.py --model_name_or_path diffusers/stable-diffusion-xl-1.0-inpainting-0.1 --base_image https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/inpaint.png --mask_image https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/inpaint_mask.png --prompts "concept art digital painting of an elven castle, inspired by lord of the rings, highly detailed, 8k" --seed 42 --scheduler euler_discrete --num_images_per_prompt 12 --batch_size 4 --image_save_dir ./xl2 --use_habana --use_hpu_graphs --gaudi_config Habana/stable-diffusion --sdp_on_bf16 --bf16
Before:
After:
![image](https://private-user-images.githubusercontent.com/34643241/408579529-509f921a-c0a2-48bb-a429-aa5b2ef5ef7a.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MzkwOTA4MzQsIm5iZiI6MTczOTA5MDUzNCwicGF0aCI6Ii8zNDY0MzI0MS80MDg1Nzk1MjktNTA5ZjkyMWEtYzBhMi00OGJiLWE0MjktYWE1YjJlZjVlZjdhLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTAyMDklMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUwMjA5VDA4NDIxNFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTkzY2E2NDM0ZmVhMWI1YWRhN2Q1MGE3Yjg2MzMxNWY1M2I1NmE4MmVjNGNlZDQxYzQxY2Y4YWZkOTViOTU1MDcmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.BlVbyutP2CZfb-mPPEtOT42ZpzKJ56LVXfjwkWQZvAI)
![image_1](https://private-user-images.githubusercontent.com/34643241/408579674-a8405359-481c-441e-b305-9d2eab6f7d0f.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MzkwOTA4MzQsIm5iZiI6MTczOTA5MDUzNCwicGF0aCI6Ii8zNDY0MzI0MS80MDg1Nzk2NzQtYTg0MDUzNTktNDgxYy00NDFlLWIzMDUtOWQyZWFiNmY3ZDBmLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTAyMDklMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUwMjA5VDA4NDIxNFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTAzNjk5OGEzNGQ5N2Q0ODljMzg0NDZhZWU0ZTdkMTAyYjc1NzA0NzZkNDFmYzZmYzliMGQwNThlZDczNzY0MjQmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.zWFpkDWkHC3R1HF3YGXCDbmon1_KeJSM7fwzTU4XMO8)
![image_2](https://private-user-images.githubusercontent.com/34643241/408579743-42232bef-c424-4ea2-b8c5-45ff713013ca.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MzkwOTA4MzQsIm5iZiI6MTczOTA5MDUzNCwicGF0aCI6Ii8zNDY0MzI0MS80MDg1Nzk3NDMtNDIyMzJiZWYtYzQyNC00ZWEyLWI4YzUtNDVmZjcxMzAxM2NhLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTAyMDklMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUwMjA5VDA4NDIxNFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWRmNTI2MTNjNjM4OTEwNDRjYjljYWZiMjBhM2NjMjJhMDc1NWUyNTg1YTk2YTUzYmY2ODFmNGQ2NTZlYzhhMWImWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.9aJNFzC52NlTBmZW6kn2KWtKnd8B1n-p1Ve38z48rzQ)
![image_3](https://private-user-images.githubusercontent.com/34643241/408579795-3eeb0d8b-5408-4459-b3fb-991c0352b659.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MzkwOTA4MzQsIm5iZiI6MTczOTA5MDUzNCwicGF0aCI6Ii8zNDY0MzI0MS80MDg1Nzk3OTUtM2VlYjBkOGItNTQwOC00NDU5LWIzZmItOTkxYzAzNTJiNjU5LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTAyMDklMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUwMjA5VDA4NDIxNFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTRjOWNjNmFlMWY4YjVhNjVkZDVkNTg3NWQ4NDMzM2U1YWVhOWE2Y2Q2YWJmM2EzMTdlY2E5M2E0NWRhMWM1Y2YmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.2QXJuCWi2ohpM83sukNwP8XPtMEgjOj3eAGsc14M36A)
![image_4](https://private-user-images.githubusercontent.com/34643241/408579863-00fcfe4c-e324-4cd3-8dcd-3a7dd652fd70.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MzkwOTA4MzQsIm5iZiI6MTczOTA5MDUzNCwicGF0aCI6Ii8zNDY0MzI0MS80MDg1Nzk4NjMtMDBmY2ZlNGMtZTMyNC00Y2QzLThkY2QtM2E3ZGQ2NTJmZDcwLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTAyMDklMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUwMjA5VDA4NDIxNFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWVjZDQ4YTFlY2MwOWZkMzMyZWRiODgyNTNkOGQ1MzA0YzVkMGRiYjcyOGFmMzg5NTViMzQ5OWNmY2U4NDI1YWYmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.0uYqAuHcMTuFCiIbhOMofjcOkjboxLxFjoTJdqIHJKE)
Images(only attach first four images ):
Seeing from the results, the performance has no decline. The output pictures are more fine
Fixes # (issue)
Before submitting