Skip to content

[Olmo2]: Add Support for Olmo2 CausalLM Model in QEff #509

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

[Olmo2]: Add Support for Olmo2 CausalLM Model in QEff #509

wants to merge 1 commit into from
+413 −0

Conversation

vbaddi
Copy link
Contributor

@vbaddi vbaddi commented Jul 9, 2025 •
edited
Loading

  • Add support for Olmo2 CausalLM Model.
  • Verified on "allenai/OLMo-2-0425-1B", PL32, CL128

image

@vbaddi vbaddi self-assigned this Jul 9, 2025
@vbaddi vbaddi added the enhancement New feature or request label Jul 9, 2025
@vbaddi vbaddi added the 1.21.0 label Jul 9, 2025
@vbaddi
Copy link
Contributor Author

vbaddi commented Sep 5, 2025

Perplexity is verified. Thanks @tchawada
Good to merge now: @quic-hemagnih

image

quic-hemagnih
quic-hemagnih approved these changes Sep 5, 2025
View reviewed changes
Copy link
Contributor

@quic-hemagnih quic-hemagnih left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will merge once CI passes

quic-rishinr
quic-rishinr approved these changes Sep 16, 2025
View reviewed changes
Copy link
Contributor

@quic-rishinr quic-rishinr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Do add the model under validated model list.

@vbaddi vbaddi force-pushed the add_olmo2 branch 3 times, most recently from dbf736c to 82de9ac Compare September 29, 2025 09:10
abukhoy
abukhoy reviewed Sep 30, 2025
View reviewed changes
abukhoy
abukhoy reviewed Oct 3, 2025
View reviewed changes
tests/transformers/models/test_causal_lm_models.py
)
assert (pytorch_hf_tokens == pytorch_kv_tokens).all(), (
"Tokens don't match for HF PyTorch model output and KV PyTorch model output"
)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove this check also, as you can see, we are checking the same in the if condition above for non-swiftkv models.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@quic-hemagnih quic-hemagnih marked this pull request as draft October 10, 2025 17:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@quic-rishinr quic-rishinr quic-rishinr approved these changes

@quic-hemagnih quic-hemagnih quic-hemagnih approved these changes

@ochougul ochougul Awaiting requested review from ochougul ochougul is a code owner

@quic-amitraj quic-amitraj Awaiting requested review from quic-amitraj quic-amitraj is a code owner

+2 more reviewers

@vinayak618 vinayak618 vinayak618 left review comments

@abukhoy abukhoy abukhoy left review comments

Reviewers whose approvals may not affect merge requirements
Assignees

@vbaddi vbaddi

Labels
1.21.0 enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@vbaddi @vinayak618 @quic-rishinr @abukhoy @quic-hemagnih