Fix recursive TransformerBridge state dict composition - #1661
Merged
jlarson4 merged 2 commits intoAug 14, 2026
Merged
Conversation
jlarson4
reviewed
Aug 13, 2026
Collaborator
|
Thanks for updating this @emerardd! This is approved, I will merge once it passes CI |
10 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This is the state-dict contract half of #1655, following the scope agreed in the maintainer discussion.
When a
TransformerBridgeis nested inside anothernn.Module, PyTorch supplies a shareddestinationand a childprefixwhile recursively collecting state. The Bridge override instead delegated to the raw Hugging Face model and returned a different mapping, so parent checkpoints did not follow normalnn.Modulecomposition semantics. Switching to standard recursive traversal also exposed two existing asymmetries: joint QKV/gate-up save hooks removed registered child keys without load-side handling, and the first forward registered_ln1_module, changing the module tree after a checkpoint had been saved.This PR:
bridge.state_dict()TransformerLens key contract;nn.Module.state_dict()semantics wheneverdestinationorprefixis supplied;Part of #1655. The BERT adapter mappings, container-buffer ownership, and broader architecture traversal sweep remain isolated to the follow-up PR requested by the maintainer.
Type of change
Validation
5266 passed, 55 skipped, 54 deselected, 10 xfailedin 14m21s51 passedmypy .: success across 431 source filesgit diff --check: cleanChecklist