Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix qwen2 vl failure in intel cpu and xpu #2709

Closed
wants to merge 1 commit into from

Conversation

sywangyi
Copy link
Contributor

@sywangyi sywangyi commented Oct 31, 2024

Hi, I see your guys have added qwen2 vl into tgi, and I try the model in intel cpu, and find it does not work, failure like

data: {"error":"Request failed during generation: Server error: output with shape [1] doesn't match the broadcast shape [873]","error_type":"generation"}

I debug it and find following code is missing in modeling and cause prefill output shape is not as expected.

@drbh @Narsil please help review it

@sywangyi sywangyi changed the title fix qwen2 failure in intel cpu fix qwen2 vl failure in intel cpu Oct 31, 2024
@sywangyi sywangyi changed the title fix qwen2 vl failure in intel cpu fix qwen2 vl failure in intel cpu and xpu Oct 31, 2024
@drbh
Copy link
Collaborator

drbh commented Oct 31, 2024

hi @sywangyi thank you for sharing this fix! I've included these changes in another PR that includes a couple other tweaks/fixes to qwen2-vl. #2708. Once that PR is merged I'll close this one. Thank you again for the fix! 🙏

@sywangyi
Copy link
Contributor Author

sywangyi commented Nov 1, 2024

2708 is merged. so close the pr

@sywangyi sywangyi closed this Nov 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants