-
Notifications
You must be signed in to change notification settings - Fork 419
[Feature][Bugfix] Add CFG feature to Bagel #1310
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
@princepride PTAL |
fa0e4e4 to
ba55194
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ba551948a9
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
|
PTAL!😊❤️ @princepride @hsliuustc0106 I will revise the stage configuration and comments in a short time, taking your feedback into account! |
|
@nussejzz Can you revert the change of load_weight |
|
@princepride Thanks for your help! I will learn from it!😊 |
Signed-off-by: Ding Zuhao <[email protected]>
Signed-off-by: Ding Zuhao <[email protected]>
Signed-off-by: Ding Zuhao <[email protected]>
Signed-off-by: Ding Zuhao <[email protected]>
Signed-off-by: Ding Zuhao <[email protected]>
Signed-off-by: Ding Zuhao <[email protected]>
…er logic Signed-off-by: Ding Zuhao <[email protected]>
Signed-off-by: Ding Zuhao <[email protected]>
Signed-off-by: Ding Zuhao <[email protected]>
Removed comment about CFG parameters. Signed-off-by: 汪志鹏 <[email protected]>
Signed-off-by: 汪志鹏 <[email protected]>
Signed-off-by: 汪志鹏 <[email protected]>
Signed-off-by: 汪志鹏 <[email protected]>
Co-authored-by: Wang Zhipeng <[email protected]> Signed-off-by: Ding Zuhao <[email protected]>
PLEASE FILL IN THE PR DESCRIPTION HERE ENSURING ALL CHECKLIST ITEMS (AT THE BOTTOM) HAVE BEEN CONSIDERED.
Purpose
This PR fixes the noisy image issue in img2img generation by implementing Classifier-Free Guidance (CFG) for both text and image conditions.
#1047
Key Changes:
✅ Implemented dual CFG support (text CFG + image CFG) in Bagel transformer
✅ Added configurable CFG scales via command-line arguments (--cfg-text-scale, --cfg-img-scale)
✅ Improved image resizing logic to ensure proper stride alignment
✅ Refactored weight loading logic for better compatibility
Test Plan
The four modes of offline inference were run, with a focus on comparing the changes in image editing and text-based image tasks.
Test Result
It ran successfully, and the quality of the generated images has been greatly improved.
Image Edit
prompt: "let the dog open half of its eyes"
original:

without cfg or disable cfg:

with cfg:

python end2end.py --modality img2img --image-path puppy.jpg --prompts "let the dog open half of its eyes" --cfg-text-scale 3.0 --cfg-img-scale 1.2 --steps 60Text2img
--prompts "A woman drawing a blue dress is dancing happily"

--cfg-text-scale 3.0
Essential Elements of an Effective PR Description Checklist
supported_models.mdandexamplesfor a new model.BEFORE SUBMITTING, PLEASE READ https://github.com/vllm-project/vllm-omni/blob/main/CONTRIBUTING.md (anything written below this line will be removed by GitHub Actions)