Skip to content

add 3 more small models#390

Merged
mike-ferguson merged 2 commits intomainfrom
add_3_small_models
Mar 12, 2026
Merged

add 3 more small models#390
mike-ferguson merged 2 commits intomainfrom
add_3_small_models

Conversation

@mike-ferguson
Copy link
Member

Add TinyLlama 1.1B, Pythia 2.8B, and OPT 2.7B model plugins

Changes

  • TinyLlama 1.1B (brainscore_language/models/tinyllama1_1b/): Registers TinyLlama/TinyLlama-1.1B-Chat-v1.0 via HuggingfaceSubject, mapped to model.layers.21 (last of 22 layers, hidden size 2048).
  • Pythia 2.8B (brainscore_language/models/pythia2_8b/): Registers EleutherAI/pythia-2.8b via HuggingfaceSubject, mapped to gpt_neox.layers.31 (last of 32 layers, hidden size 2560).
  • OPT 2.7B (brainscore_language/models/opt2_7b/): Registers facebook/opt-2.7b via HuggingfaceSubject, mapped to model.decoder.layers.31 (last of 32 layers, hidden size 2560).
  • Modifies the test_identifier test in phi-2 to be correct.

Each plugin includes tests for model loading, identifier, neural representations and next-word prediction.

Notes

  • All three are open (non-gated) models — no HuggingFace token required.
  • Layer mappings are defaults; can be updated after benchmark-driven selection.

@mike-ferguson mike-ferguson added the submission_prepared Attached to a PR is metadata and layer mapping is successful. label Mar 11, 2026
@mike-ferguson
Copy link
Member Author

tests are failing due to OOM, to be expected. Manual merge

@mike-ferguson mike-ferguson merged commit d8f55f4 into main Mar 12, 2026
10 of 11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

submission_prepared Attached to a PR is metadata and layer mapping is successful.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant