diff --git a/docs/source/content/model_structure.md b/docs/source/content/model_structure.md
new file mode 100644
index 000000000..0ab0ff4a3
--- /dev/null
+++ b/docs/source/content/model_structure.md
@@ -0,0 +1,154 @@
+# TransformerBridge Model Structure
+
+This page describes the structure exposed by TransformerBridge, the canonical hook names to use, and the expected tensor shapes at each hook point.
+
+## Overview
+
+TransformerBridge wraps a Hugging Face model behind a consistent TransformerLens interface. It relies on:
+- An ArchitectureAdapter that understands the HF module graph and provides a mapping to bridge components
+- Generalized components (Embedding, Attention, MLP, Normalization, Block) exposing uniform hook points
+- A light aliasing layer for backwards compatibility with legacy TransformerLens hook names
+
+Construct a bridge from a HF model id:
+
+```python
+from transformer_lens.model_bridge.bridge import TransformerBridge
+from transformer_lens.model_bridge.sources import transformers as bridge_sources  # registers boot
+
+bridge = TransformerBridge.boot_transformers("gpt2", device="cpu")
+```
+
+You can then call the familiar APIs: `to_tokens`, `to_string`, `generate`, `run_with_hooks`, `run_with_cache`.
+
+## Top-Level Components
+
+Typical decoder-only models expose these top-level components (names vary by architecture):
+- `embed`: token embedding
+- `pos_embed` (if applicable) or rotary embeddings inside attention
+- `blocks`: list-like container of transformer blocks
+- `ln_final` (if applicable): final normalization
+- `unembed`: output projection to vocabulary logits
+
+Each `blocks.{i}` is a `BlockBridge` with subcomponents:
+- `ln1`: normalization before attention
+- `attn`: attention module
+- `ln2`: normalization before MLP
+- `mlp`: MLP module
+
+## Canonical Hook Names
+
+Use these canonical (non-aliased) names when adding hooks or reading from the cache.
+
+### Embedding
+- `embed.hook_in`: token ids (batch, pos)
+- `embed.hook_out`: embeddings (batch, pos, d_model)
+  - *Legacy alias: `hook_embed`*
+- `pos_embed.hook_in` / `pos_embed.hook_out`: same shapes as above
+  - *Legacy alias: `hook_pos_embed`*
+
+### Residual stream
+- `blocks.{i}.hook_in`: residual stream into block (batch, pos, d_model)
+  - *Legacy alias: `blocks.{i}.hook_resid_pre`*
+- `blocks.{i}.hook_out`: residual stream out of block (batch, pos, d_model)
+  - *Legacy alias: `blocks.{i}.hook_resid_post`*
+- `blocks.{i}.attn.hook_out`: residual stream after attention (batch, pos, d_model)
+  - *Legacy alias: `blocks.{i}.hook_resid_mid`*
+
+### Attention
+- `blocks.{i}.attn.hook_in`: (batch, pos, d_model)
+  - *Legacy alias: `blocks.{i}.hook_attn_in`*
+- `blocks.{i}.attn.hook_out`: (batch, pos, d_model)
+  - *Legacy alias: `blocks.{i}.hook_attn_out`*
+- `blocks.{i}.attn.hook_hidden_states`: primary output for caching (batch, pos, d_model)
+  - *Legacy alias: `blocks.{i}.attn.hook_result`*
+- `blocks.{i}.attn.hook_attn_scores`: raw attention scores before softmax (batch, n_heads, pos, pos)
+- `blocks.{i}.attn.hook_pattern`: attention pattern after softmax and NaN handling (n_heads, pos, pos)
+  - *Legacy alias: `blocks.{i}.attn.hook_attention_weights`*
+- When present, sub-projections: `blocks.{i}.attn.q/k/v/o.hook_in` / `.hook_out` (commonly (batch, pos, d_model))
+  - *Legacy aliases: `blocks.{i}.hook_q_input`, `blocks.{i}.hook_k_input`, `blocks.{i}.hook_v_input`, `blocks.{i}.hook_q`, `blocks.{i}.hook_k`, `blocks.{i}.hook_v`*
+
+#### Individual Q/K/V Hooks
+All attention bridges provide access to individual Q, K, V activations through `HookPointWrapper` properties:
+
+- `blocks.{i}.attn.q.hook_in` / `blocks.{i}.attn.q.hook_out`: Q projection hooks (batch, pos, n_heads, d_head)
+- `blocks.{i}.attn.k.hook_in` / `blocks.{i}.attn.k.hook_out`: K projection hooks (batch, pos, n_heads, d_head)
+- `blocks.{i}.attn.v.hook_in` / `blocks.{i}.attn.v.hook_out`: V projection hooks (batch, pos, n_heads, d_head)
+
+#### Joint QKV Attention (GPT-2 style)
+For models using fused QKV projections (like GPT-2), the `JointQKVAttentionBridge` provides additional hooks:
+
+- `blocks.{i}.attn.qkv.hook_in`: input to QKV projection (batch, pos, d_model)
+- `blocks.{i}.attn.qkv.hook_out`: output from QKV projection (batch, pos, 3*d_model)
+- `blocks.{i}.attn.qkv.q_hook_in`: input to Q projection (batch, pos, d_model)
+- `blocks.{i}.attn.qkv.q_hook_out`: output from Q projection (batch, pos, n_heads, d_head)
+- `blocks.{i}.attn.qkv.k_hook_in`: input to K projection (batch, pos, d_model)
+- `blocks.{i}.attn.qkv.k_hook_out`: output from K projection (batch, pos, n_heads, d_head)
+- `blocks.{i}.attn.qkv.v_hook_in`: input to V projection (batch, pos, d_model)
+- `blocks.{i}.attn.qkv.v_hook_out`: output from V projection (batch, pos, n_heads, d_head)
+
+### MLP
+- `blocks.{i}.mlp.hook_in`: (batch, pos, d_model)
+  - *Legacy alias: `blocks.{i}.hook_mlp_in`*
+- `blocks.{i}.mlp.hook_pre`: (batch, pos, d_mlp)
+  - *Legacy alias: `blocks.{i}.hook_mlp_in` (via `mlp.in.hook_out`)*
+- `blocks.{i}.mlp.hook_out`: (batch, pos, d_model)
+  - *Legacy alias: `blocks.{i}.hook_mlp_out`*
+
+### Normalization
+- `blocks.{i}.ln1.hook_in` / `.hook_out`: (batch, pos, d_model)
+  - *Legacy aliases for `.hook_out`: `blocks.{i}.ln1.hook_normalized`, `blocks.{i}.ln1.hook_scale`*
+- Similarly for `ln2`
+  - *Legacy aliases for `.hook_out`: `blocks.{i}.ln2.hook_normalized`, `blocks.{i}.ln2.hook_scale`*
+
+### Unembedding / Logits
+- `unembed.hook_in`: (batch, pos, d_model)
+- `unembed.hook_out`: (batch, pos, d_vocab)
+
+## Shapes at a Glance
+
+- Residual stream and hidden states: (batch, pos, d_model)
+- Attention scores: (batch, n_heads, pos, pos)
+- Attention patterns: (n_heads, pos, pos) - after batch dimension removal
+- QKV projections: (batch, pos, n_heads, d_head)
+- MLP pre-activation: (batch, pos, d_mlp)
+- Embeddings: (batch, pos, d_model)
+- Unembedding logits: (batch, pos, d_vocab)
+- LayerNorm normalized / scale: (batch, pos, d_model)
+
+These shapes are exercised in the multi-model shape test: `tests/integration/test_hook_shape_compatibility.py`.
+
+## Booting from Hugging Face
+
+`TransformerBridge.boot_transformers(model_id, ...)`:
+- Loads the HF config/model/tokenizer
+- Selects the appropriate ArchitectureAdapter
+- Maps HF config fields to TransformerLens config (e.g., `d_model`, `n_heads`, `n_layers`, `d_mlp`, `d_vocab`, `n_ctx`, ...)
+- Constructs the bridge and registers all hook points
+
+## Fused QKV Attention
+
+Some architectures use a fused QKV projection (like GPT-2). The bridge's `JointQKVAttentionBridge` provides access to individual Q, K, V activations through the `QKVBridge` submodule. This allows for:
+
+1. **Individual Q/K/V hooking**: You can hook into `blocks.{i}.attn.qkv.q_hook_out`, `k_hook_out`, or `v_hook_out` to modify individual attention heads
+2. **Attention pattern creation**: The bridge automatically creates attention patterns from the attention scores and applies them through `hook_pattern`
+3. **Compatibility with legacy code**: Legacy hook names like `blocks.{i}.hook_v` are aliased to the appropriate QKV hooks
+
+The canonical attention hooks (`attn.hook_in/out`, `attn.hook_pattern`, etc.) retain the shapes listed above, while the QKV-specific hooks provide access to the individual attention components.
+
+## Aliases and Backwards Compatibility
+
+A minimal alias layer exists to ease migration from older TransformerLens names (e.g., `blocks.{i}.hook_resid_pre` → `blocks.{i}.hook_in`). New code should prefer the canonical names documented here.
+
+## Example: Caching and Inspecting Hooks
+
+```python
+prompt = "Hello world"
+logits, cache = bridge.run_with_cache(prompt)
+
+# List some attention-related hooks on the first block
+for k in cache.keys():
+    if k.startswith("blocks.0.attn"):
+        print(k, cache[k].shape)
+```
+
+For larger examples and a multi-model shape check, see `tests/integration/test_hook_shape_compatibility.py`.
diff --git a/tests/integration/test_hook_shape_compatibility.py b/tests/integration/test_hook_shape_compatibility.py
new file mode 100644
index 000000000..c8b8db215
--- /dev/null
+++ b/tests/integration/test_hook_shape_compatibility.py
@@ -0,0 +1,238 @@
+import gc
+import os
+from typing import Iterable, Tuple
+
+import pytest
+import torch
+
+
+def _to_list(keys: Iterable[str]) -> list[str]:
+    return list(keys) if not isinstance(keys, list) else keys
+
+
+# Test models selected for architectural diversity while respecting memory constraints.
+# Note: Python doesn't release model memory between parameterized tests, causing accumulation.
+# Models are ordered by size to minimize peak memory usage.
+PUBLIC_HF_MODELS = [
+    "sshleifer/tiny-gpt2",
+    "EleutherAI/pythia-70m",
+    "facebook/opt-125m",
+]
+
+# Extended model list for authenticated CI runs
+FULL_HF_MODELS = [
+    "sshleifer/tiny-gpt2",
+    "EleutherAI/pythia-70m",
+    "roneneldan/TinyStories-33M",
+    "facebook/opt-125m",
+    "gpt2",
+]
+
+
+def _select_model_ids_from_acceptance_lists() -> list[str]:
+    return FULL_HF_MODELS if os.environ.get("HF_TOKEN", "") else PUBLIC_HF_MODELS
+
+
+# Allow overriding via env, comma-separated HF ids
+DEFAULT_IDS = ",".join(_select_model_ids_from_acceptance_lists())
+MODELS_ENV = os.getenv("TL_HOOK_SHAPE_MODELS", DEFAULT_IDS)
+MODEL_NAMES = [m.strip() for m in MODELS_ENV.split(",") if m.strip()]
+
+
+def _expected_shape_for_name(
+    name: str,
+    *,
+    batch: int,
+    pos: int,
+    d_model: int,
+    d_vocab: int | None,
+    n_heads: int | None,
+    d_head: int | None,
+    d_mlp: int | None,
+) -> Tuple[int, ...] | None:
+    # Canonical TransformerBridge hook names only (no legacy aliases)
+
+    # Unembedding (check before embedding to avoid matching "embed" in "unembed")
+    if name.endswith("unembed.hook_in"):
+        return (batch, pos, d_model)
+    if name.endswith("unembed.hook_out") and d_vocab is not None:
+        return (batch, pos, d_vocab)
+
+    # Embedding components
+    if name.endswith("embed.hook_in") or name.endswith("pos_embed.hook_in"):
+        return (batch, pos)
+    if name.endswith("embed.hook_out") or name.endswith("pos_embed.hook_out"):
+        return (batch, pos, d_model)
+
+    # Block IO
+    if ".hook_in" in name and ".attn." not in name and ".mlp." not in name and ".ln" not in name:
+        # blocks.{i}.hook_in
+        return (batch, pos, d_model)
+    if ".hook_out" in name and ".attn." not in name and ".mlp." not in name and ".ln" not in name:
+        # blocks.{i}.hook_out
+        return (batch, pos, d_model)
+
+    # Attention module (canonical TB names)
+    if name.endswith("attn.hook_in") or name.endswith("attn.hook_out"):
+        return (batch, pos, d_model)
+    if name.endswith("attn.hook_hidden_states"):
+        return (batch, pos, d_model)
+    if name.endswith("attn.hook_attention_weights") and n_heads is not None:
+        return (batch, n_heads, pos, pos)
+    if name.endswith("attn.hook_attn_scores") and n_heads is not None:
+        return (batch, n_heads, pos, pos)
+    if name.endswith("attn.hook_pattern") and n_heads is not None:
+        return (batch, n_heads, pos, pos)
+
+    # Attention subprojections: q/k/v/o
+    # Note: q/k/v hooks can be either:
+    # - (batch, pos, n_heads, d_head) for models with split heads (GPT-2, Pythia, etc.)
+    # - (batch, pos, d_model) for models without split heads (GPT-Neo, etc.)
+    # Both are valid depending on the architecture
+    if name.endswith("attn.o.hook_in"):
+        return (batch, pos, d_model)
+    if name.endswith("attn.o.hook_out"):
+        return (batch, pos, d_model)
+
+    # LayerNorms within blocks
+    if ".ln" in name and name.endswith("hook_in"):
+        return (batch, pos, d_model)
+    if ".ln" in name and name.endswith("hook_out"):
+        return (batch, pos, d_model)
+    if name.endswith("hook_normalized"):
+        return (batch, pos, d_model)
+    if name.endswith("hook_scale"):
+        # LayerNorm scale is (batch, pos, 1) for broadcasting
+        return (batch, pos, 1)
+
+    # MLP module
+    if name.endswith("mlp.hook_in") or name.endswith("mlp.hook_out"):
+        return (batch, pos, d_model)
+    if name.endswith("mlp.hook_pre") and d_mlp is not None:
+        return (batch, pos, d_mlp)
+    # MLP submodules: input and out projections
+    if name.endswith("mlp.input.hook_in") or name.endswith("mlp.out.hook_out"):
+        return (batch, pos, d_model)
+    if (
+        name.endswith("mlp.input.hook_out") or name.endswith("mlp.out.hook_in")
+    ) and d_mlp is not None:
+        return (batch, pos, d_mlp)
+
+    return None
+
+
+@pytest.mark.parametrize("model_name", MODEL_NAMES)
+def test_transformer_bridge_hook_shapes(model_name: str):
+    # Ensure boot method is registered
+    from transformer_lens.model_bridge.bridge import TransformerBridge
+    from transformer_lens.model_bridge.sources import (  # noqa: F401
+        transformers as bridge_sources,
+    )
+
+    bridge = TransformerBridge.boot_transformers(model_name, device="cpu")
+
+    prompt = "Hello world"
+    tokens = bridge.to_tokens(prompt, move_to_device=False)
+    batch, pos = int(tokens.shape[0]), int(tokens.shape[1])
+
+    cfg = bridge.cfg
+    d_model = int(getattr(cfg, "d_model"))
+    # Use actual vocab size from weights (may differ from config due to padding)
+    d_vocab = None
+    if hasattr(bridge, "unembed") and hasattr(bridge.unembed, "weight"):
+        d_vocab = int(bridge.unembed.weight.shape[0])
+    elif hasattr(cfg, "d_vocab"):
+        d_vocab = int(getattr(cfg, "d_vocab", 0))
+
+    n_heads = int(getattr(cfg, "n_heads", 0)) if hasattr(cfg, "n_heads") else None
+    d_head = int(getattr(cfg, "d_head", 0)) if hasattr(cfg, "d_head") else None
+    d_mlp = int(getattr(cfg, "d_mlp", 0)) if hasattr(cfg, "d_mlp") else None
+    if n_heads == 0:
+        n_heads = None
+    if d_head == 0:
+        d_head = None
+    if d_mlp == 0:
+        d_mlp = None
+
+    _, cache = bridge.run_with_cache(tokens, device="cpu")
+    keys = sorted(_to_list(cache.keys()))
+
+    # OPT reshapes to (batch*seq, d_model) internally for efficiency
+    is_opt_model = "opt" in model_name.lower()
+
+    mismatches: list[tuple[str, Tuple[int, ...], Tuple[int, ...]]] = []
+    checked = 0
+    for name in keys:
+        # Special handling for q/k/v hooks which can have two valid shapes
+        is_qkv_hook = any(
+            name.endswith(suf)
+            for suf in (
+                "attn.q.hook_in",
+                "attn.k.hook_in",
+                "attn.v.hook_in",
+                "attn.q.hook_out",
+                "attn.k.hook_out",
+                "attn.v.hook_out",
+            )
+        )
+
+        if is_qkv_hook:
+            tensor = cache[name]
+            assert isinstance(tensor, torch.Tensor), f"Non-tensor cached for {name}"
+            got = tuple(tensor.shape)
+            # Valid shapes: (batch, pos, n_heads, d_head) or (batch, pos, d_model)
+            valid_shapes = []
+            if n_heads is not None and d_head is not None:
+                valid_shapes.append((batch, pos, n_heads, d_head))
+            valid_shapes.append((batch, pos, d_model))
+
+            if got not in valid_shapes:
+                exp_str = " or ".join(str(s) for s in valid_shapes)
+                mismatches.append((name, exp_str, got))  # type: ignore
+            checked += 1
+            continue
+
+        # Rotary embeddings have architecture-specific partial dimensions
+        if "rotary" in name.lower():
+            checked += 1
+            continue
+
+        exp = _expected_shape_for_name(
+            name,
+            batch=batch,
+            pos=pos,
+            d_model=d_model,
+            d_vocab=d_vocab,
+            n_heads=n_heads,
+            d_head=d_head,
+            d_mlp=d_mlp,
+        )
+        if exp is None:
+            continue
+        tensor = cache[name]
+        assert isinstance(tensor, torch.Tensor), f"Non-tensor cached for {name}"
+        got = tuple(tensor.shape)
+
+        # OPT flattens batch and sequence dimensions for MLP/LayerNorm
+        if is_opt_model and got != exp:
+            is_flattened_hook = (".ln" in name and ".hook" in name and ".attn.ln" not in name) or (
+                ".mlp." in name and "hook" in name
+            )
+
+            if is_flattened_hook and len(exp) == 3 and len(got) == 2:
+                if got == (batch * pos, exp[2]):
+                    checked += 1
+                    continue
+
+        if got != exp:
+            mismatches.append((name, exp, got))
+        checked += 1
+
+    assert checked > 0, "No hooks were checked; update expected mapping or model filter"
+    msg = "\n".join(f"{n}: expected {e}, got {g}" for n, e, g in mismatches[:20])
+    assert not mismatches, f"Found {len(mismatches)} shape mismatches. Examples:\n{msg}"
+
+    # Clean up to reduce memory usage during parameterized test runs
+    del bridge, cache, tokens
+    torch.cuda.empty_cache() if torch.cuda.is_available() else None
+    gc.collect()
diff --git a/transformer_lens/model_bridge/architecture_adapter.py b/transformer_lens/model_bridge/architecture_adapter.py
index fb7a85249..b16279a45 100644
--- a/transformer_lens/model_bridge/architecture_adapter.py
+++ b/transformer_lens/model_bridge/architecture_adapter.py
@@ -156,7 +156,10 @@ def get_component_from_list_module(
                 if len(parts) > 3:
                     # Navigate through the deeper subcomponents
                     current_bridge = subcomponent_bridge
-                    current = getattr(item, subcomponent_bridge.name)
+                    if subcomponent_bridge.name is None:
+                        current = item
+                    else:
+                        current = getattr(item, subcomponent_bridge.name)
 
                     for i in range(3, len(parts)):
                         deeper_component_name = parts[i]
@@ -171,7 +174,11 @@ def get_component_from_list_module(
                         # Check submodules for deeper components
                         if deeper_component_name in current_bridge.submodules:
                             current_bridge = current_bridge.submodules[deeper_component_name]
-                            current = getattr(current, current_bridge.name)
+                            if current_bridge.name is None:
+                                # No container, stay at current level
+                                pass
+                            else:
+                                current = getattr(current, current_bridge.name)
                         else:
                             raise ValueError(
                                 f"Component {deeper_component_name} not found in {'.'.join(parts[:i])} components"
@@ -180,7 +187,10 @@ def get_component_from_list_module(
                     return current
                 else:
                     # Just the 3-level path
-                    return getattr(item, subcomponent_bridge.name)
+                    if subcomponent_bridge.name is None:
+                        return item
+                    else:
+                        return getattr(item, subcomponent_bridge.name)
             else:
                 raise ValueError(
                     f"Component {subcomponent_name} not found in {parts[0]} components"
@@ -338,16 +348,22 @@ def get_component(self, model: RemoteModel, path: TransformerLensPath) -> Remote
 
         if len(parts) == 1:
             # Simple case: just return the component at the bridge's remote path
+            if bridge_component.name is None:
+                return model
             return self.get_remote_component(model, bridge_component.name)
 
         # For nested paths like "blocks.0.attn", we need to handle the indexing
         if bridge_component.is_list_item and len(parts) >= 2:
             # Get the remote ModuleList for the indexed item
+            if bridge_component.name is None:
+                raise ValueError(f"List component {parts[0]} must have a name")
             list_module = self.get_remote_component(model, bridge_component.name)
             return self.get_component_from_list_module(list_module, bridge_component, parts)
 
         # For other nested paths, navigate through the remote model
         remote_path = bridge_component.name
+        if remote_path is None:
+            raise ValueError(f"Component {parts[0]} must have a name for nested paths")
         if len(parts) > 1:
             remote_path = f"{remote_path}.{'.'.join(parts[1:])}"
 
@@ -389,6 +405,8 @@ def translate_transformer_lens_path(
         if len(parts) == 1:
             # Simple case: just return the bridge's remote path
             remote_path = bridge_component.name
+            if remote_path is None:
+                raise ValueError(f"Component {parts[0]} must have a name for path translation")
             # Add parameter suffix from preprocessing
             if param_suffix:
                 remote_path = remote_path + param_suffix
@@ -405,6 +423,8 @@ def translate_transformer_lens_path(
 
             # Get the base items path
             items_path = bridge_component.name
+            if items_path is None:
+                raise ValueError(f"List component {parts[0]} must have a name for path translation")
 
             if len(parts) == 2:
                 # Just return the indexed item path
@@ -427,7 +447,12 @@ def translate_transformer_lens_path(
                     if len(parts) > 3:
                         # Navigate through the deeper subcomponents
                         current_bridge = subcomponent_bridge
-                        remote_path_parts = [items_path, item_index, subcomponent_bridge.name]
+                        subcomponent_name_str = subcomponent_bridge.name
+                        if subcomponent_name_str is None:
+                            raise ValueError(
+                                f"Subcomponent {subcomponent_name} must have a name for path translation"
+                            )
+                        remote_path_parts = [items_path, item_index, subcomponent_name_str]
 
                         for i in range(3, len(parts)):
                             deeper_component_name = parts[i]
@@ -435,7 +460,12 @@ def translate_transformer_lens_path(
                             # Check submodules for deeper components
                             if deeper_component_name in current_bridge.submodules:
                                 current_bridge = current_bridge.submodules[deeper_component_name]
-                                remote_path_parts.append(current_bridge.name)
+                                deeper_name = current_bridge.name
+                                if deeper_name is None:
+                                    raise ValueError(
+                                        f"Component {deeper_component_name} must have a name for path translation"
+                                    )
+                                remote_path_parts.append(deeper_name)
                             else:
                                 raise ValueError(
                                     f"Component {deeper_component_name} not found in {'.'.join(parts[:i])} components"
@@ -450,7 +480,12 @@ def translate_transformer_lens_path(
                         return remote_path
                     else:
                         # Just the 3-level path
-                        remote_path = f"{items_path}.{item_index}.{subcomponent_bridge.name}"
+                        subcomponent_name_str = subcomponent_bridge.name
+                        if subcomponent_name_str is None:
+                            raise ValueError(
+                                f"Subcomponent {subcomponent_name} must have a name for path translation"
+                            )
+                        remote_path = f"{items_path}.{item_index}.{subcomponent_name_str}"
                         # Add parameter suffix from preprocessing
                         if param_suffix:
                             remote_path = remote_path + param_suffix
@@ -464,6 +499,8 @@ def translate_transformer_lens_path(
 
         # For other nested paths, navigate through the bridge components
         remote_path = bridge_component.name
+        if remote_path is None:
+            raise ValueError(f"Component {parts[0]} must have a name for path translation")
         if len(parts) > 1:
             remote_path = f"{remote_path}.{'.'.join(parts[1:])}"
 
@@ -836,9 +873,12 @@ def extract_weights_using_components(self, model) -> dict[str, torch.Tensor]:
                         elif subcomp_name == "attn":
                             # Attention component needs config and split function (if it's a JointQKVAttentionBridge)
                             if issubclass(component_class, JointQKVAttentionBridge):
+                                attn_name = subcomponent.name
+                                if attn_name is None:
+                                    raise ValueError("Attention component must have a name")
                                 if hasattr(self, "split_qkv_matrix"):
                                     fresh_component = component_class(
-                                        name=subcomponent.name,
+                                        name=attn_name,
                                         config=self.cfg,
                                         split_qkv_matrix=self.split_qkv_matrix,
                                     )
@@ -848,7 +888,7 @@ def dummy_split_qkv_matrix(attn_layer):
                                         return None, None, None
 
                                     fresh_component = component_class(
-                                        name=subcomponent.name,
+                                        name=attn_name,
                                         config=self.cfg,
                                         split_qkv_matrix=dummy_split_qkv_matrix,
                                     )
@@ -915,8 +955,13 @@ def process_weights(
                                                         ] = bias_tensor.clone()
                                                     self._processed_weights = processed_weights
 
+                                            mlp_input_name = mlp_subcomponent.name
+                                            if mlp_input_name is None:
+                                                raise ValueError(
+                                                    "MLP input component must have a name"
+                                                )
                                             mlp_fresh_component = MLPInputLinearBridge(
-                                                name=mlp_subcomponent.name
+                                                name=mlp_input_name
                                             )
                                         elif mlp_subcomp_name == "out":
 
@@ -949,12 +994,22 @@ def process_weights(
                                                         ] = bias_tensor.clone()
                                                     self._processed_weights = processed_weights
 
+                                            mlp_output_name = mlp_subcomponent.name
+                                            if mlp_output_name is None:
+                                                raise ValueError(
+                                                    "MLP output component must have a name"
+                                                )
                                             mlp_fresh_component = MLPOutputLinearBridge(
-                                                name=mlp_subcomponent.name
+                                                name=mlp_output_name
                                             )
                                         else:
+                                            mlp_generic_name = mlp_subcomponent.name
+                                            if mlp_generic_name is None:
+                                                raise ValueError(
+                                                    f"MLP component {mlp_subcomp_name} must have a name"
+                                                )
                                             mlp_fresh_component = LinearBridge(
-                                                name=mlp_subcomponent.name
+                                                name=mlp_generic_name
                                             )
 
                                         mlp_fresh_component.set_original_component(
diff --git a/transformer_lens/model_bridge/component_setup.py b/transformer_lens/model_bridge/component_setup.py
index 99000eca7..3a7d5de38 100644
--- a/transformer_lens/model_bridge/component_setup.py
+++ b/transformer_lens/model_bridge/component_setup.py
@@ -76,31 +76,30 @@ def setup_submodules(
     for module_name, submodule in component.submodules.items():
         if submodule.is_list_item:
             # Submodule is a BlockBridge - create a ModuleList of bridge components
+            if submodule.name is None:
+                raise ValueError(f"List item component {module_name} must have a name")
             bridged_list = setup_blocks_bridge(submodule, architecture_adapter, original_model)
             # Set the list on the bridge module as a proper module
             component.add_module(module_name, bridged_list)
             replace_remote_component(bridged_list, submodule.name, original_model)
         # Only add if not already registered as a PyTorch module
         if module_name not in component._modules:
-            # Get the original component for this submodule
-            remote_path = submodule.name
+            # Get original component (use parent if no container, e.g. OPT's MLP)
+            if submodule.name is None:
+                original_subcomponent = original_model
+            else:
+                remote_path = submodule.name
+                original_subcomponent = architecture_adapter.get_remote_component(
+                    original_model, remote_path
+                )
 
-            original_subcomponent = architecture_adapter.get_remote_component(
-                original_model, remote_path
-            )
-
-            # Set the original component
             submodule.set_original_component(original_subcomponent)
-
-            # Recursively set up submodules of this submodule
             setup_submodules(submodule, architecture_adapter, original_subcomponent)
-
-            # Add the submodule to the parent component
             component.add_module(module_name, submodule)
 
-            # Replace the original submodule with the bridged submodule in the parent
-            # Use the actual component's remote name in the parent component
-            replace_remote_component(submodule, submodule.name, original_model)
+            # Replace original with bridge (skip if no container)
+            if submodule.name is not None:
+                replace_remote_component(submodule, submodule.name, original_model)
 
 
 def setup_components(
diff --git a/transformer_lens/model_bridge/generalized_components/base.py b/transformer_lens/model_bridge/generalized_components/base.py
index 3e7d0e211..42d982add 100644
--- a/transformer_lens/model_bridge/generalized_components/base.py
+++ b/transformer_lens/model_bridge/generalized_components/base.py
@@ -39,7 +39,7 @@ class GeneralizedComponent(nn.Module):
 
     def __init__(
         self,
-        name: str,
+        name: Optional[str],
         config: Optional[Any] = None,
         submodules: Optional[Dict[str, "GeneralizedComponent"]] = None,
         conversion_rule: Optional[BaseHookConversion] = None,
@@ -47,7 +47,7 @@ def __init__(
         """Initialize the generalized component.
 
         Args:
-            name: The name of this component
+            name: The name of this component (None if component has no container in remote model)
             config: Optional configuration object for the component
             submodules: Dictionary of GeneralizedComponent submodules to register
             conversion_rule: Optional conversion rule for this component's hooks
diff --git a/transformer_lens/model_bridge/generalized_components/embedding.py b/transformer_lens/model_bridge/generalized_components/embedding.py
index a07ca3e3d..047a68d03 100644
--- a/transformer_lens/model_bridge/generalized_components/embedding.py
+++ b/transformer_lens/model_bridge/generalized_components/embedding.py
@@ -115,6 +115,10 @@ def forward(
         else:
             output = self.original_component(input_ids, position_ids=position_ids, **kwargs)
 
+        # Some models return tuples; extract embeddings
+        if isinstance(output, tuple):
+            output = output[0]
+
         # Apply output hook
         output = self.hook_out(output)
 
diff --git a/transformer_lens/model_bridge/generalized_components/linear.py b/transformer_lens/model_bridge/generalized_components/linear.py
index ce90d4e25..94bbc6610 100644
--- a/transformer_lens/model_bridge/generalized_components/linear.py
+++ b/transformer_lens/model_bridge/generalized_components/linear.py
@@ -92,13 +92,14 @@ def process_weights(
             return
 
         # Determine weight keys based on component name and context
-        if "c_fc" in self.name or "input" in self.name:
+        component_name = self.name or ""
+        if "c_fc" in component_name or "input" in component_name:
             weight_key = "W_in"
             bias_key = "b_in"
-        elif "c_proj" in self.name and "mlp" in str(type(self)).lower():
+        elif "c_proj" in component_name and "mlp" in str(type(self)).lower():
             weight_key = "W_out"
             bias_key = "b_out"
-        elif "c_proj" in self.name and "attn" in str(type(self)).lower():
+        elif "c_proj" in component_name and "attn" in str(type(self)).lower():
             weight_key = "W_O"
             bias_key = "b_O"
         else:
diff --git a/transformer_lens/model_bridge/generalized_components/mlp.py b/transformer_lens/model_bridge/generalized_components/mlp.py
index 9565a6ec4..f39995534 100644
--- a/transformer_lens/model_bridge/generalized_components/mlp.py
+++ b/transformer_lens/model_bridge/generalized_components/mlp.py
@@ -37,14 +37,14 @@ class MLPBridge(GeneralizedComponent):
 
     def __init__(
         self,
-        name: str,
+        name: Optional[str],
         config: Optional[Any] = None,
         submodules: Optional[Dict[str, GeneralizedComponent]] = {},
     ):
         """Initialize the MLP bridge.
 
         Args:
-            name: The name of the component in the model
+            name: The name of the component in the model (None if no container exists)
             config: Optional configuration (unused for MLPBridge)
             submodules: Dictionary of submodules to register (e.g., gate_proj, up_proj, down_proj)
         """
diff --git a/transformer_lens/model_bridge/generalized_components/normalization.py b/transformer_lens/model_bridge/generalized_components/normalization.py
index 48a19c387..38377b50d 100644
--- a/transformer_lens/model_bridge/generalized_components/normalization.py
+++ b/transformer_lens/model_bridge/generalized_components/normalization.py
@@ -142,13 +142,14 @@ def process_weights(
             return
 
         # Determine weight keys based on component name
-        if "ln_f" in self.name or "final" in self.name:
+        component_name = self.name or ""
+        if "ln_f" in component_name or "final" in component_name:
             weight_key = "w"
             bias_key = "b"
-        elif "ln_1" in self.name:
+        elif "ln_1" in component_name:
             weight_key = "w"
             bias_key = "b"
-        elif "ln_2" in self.name:
+        elif "ln_2" in component_name:
             weight_key = "w"
             bias_key = "b"
         else:
diff --git a/transformer_lens/model_bridge/generalized_components/pos_embed.py b/transformer_lens/model_bridge/generalized_components/pos_embed.py
index 4dc170653..ac97e65b2 100644
--- a/transformer_lens/model_bridge/generalized_components/pos_embed.py
+++ b/transformer_lens/model_bridge/generalized_components/pos_embed.py
@@ -3,7 +3,6 @@
 This module contains the bridge component for positional embedding layers.
 """
 
-import inspect
 from typing import Any, Dict, Optional
 
 import torch
@@ -53,22 +52,29 @@ def W_pos(self) -> torch.Tensor:
 
     def forward(
         self,
-        input_ids: torch.Tensor,
-        position_ids: torch.Tensor | None = None,
+        *args: Any,
         **kwargs: Any,
     ) -> torch.Tensor:
         """Forward pass through the positional embedding bridge.
 
+        This method accepts variable arguments to support different architectures:
+        - Standard models (GPT-2, GPT-Neo): (input_ids, position_ids=None)
+        - OPT models: (attention_mask, past_key_values_length=0, position_ids=None)
+        - Others may have different signatures
+
         Args:
-            input_ids: Input token IDs (used to determine sequence length and batch size)
-            position_ids: Optional position IDs, if None will generate them automatically
-            **kwargs: Additional arguments
+            *args: Positional arguments forwarded to the original component
+            **kwargs: Keyword arguments forwarded to the original component
 
         Returns:
             Positional embeddings
         """
         # Check if we're using processed weights from a reference model (layer norm folding case)
         if hasattr(self, "_use_processed_weights") and self._use_processed_weights:
+            # For processed weights, we expect the standard (input_ids, position_ids) signature
+            input_ids = args[0] if args else kwargs.get("input_ids")
+            position_ids = args[1] if len(args) > 1 else kwargs.get("position_ids")
+
             # Apply input hook to input_ids (for consistency, though pos embed doesn't really use input_ids)
             input_ids = self.hook_in(input_ids)
 
@@ -95,24 +101,13 @@ def forward(
                 f"Original component not set for {self.name}. Call set_original_component() first."
             )
 
-        # Apply input hook to input_ids
-        input_ids = self.hook_in(input_ids)
-
-        # For standard positional embeddings, we need to generate position indices
-        if position_ids is None:
-            batch_size, seq_len = input_ids.shape[:2]
-            position_ids = torch.arange(seq_len, device=input_ids.device, dtype=torch.long)
-            position_ids = position_ids.unsqueeze(0).expand(batch_size, -1)
-
-        # Check if the original component supports position_ids using inspect.signature
-        sig = inspect.signature(self.original_component.forward)
-        supports_position_ids = "position_ids" in sig.parameters
+        # Apply input hook to the first argument (whatever it is - input_ids or attention_mask)
+        if args:
+            first_arg = self.hook_in(args[0])
+            args = (first_arg,) + args[1:]
 
-        if not hasattr(self.original_component, "forward") or not supports_position_ids:
-            # For simple embedding layers, call directly with position_ids
-            output = self.original_component(position_ids, **kwargs)
-        else:
-            output = self.original_component(position_ids=position_ids, **kwargs)
+        # Forward all arguments to the original component
+        output = self.original_component(*args, **kwargs)
 
         # Apply output hook
         output = self.hook_out(output)
diff --git a/transformer_lens/model_bridge/supported_architectures/opt.py b/transformer_lens/model_bridge/supported_architectures/opt.py
index 311e9ef15..a44b91b0a 100644
--- a/transformer_lens/model_bridge/supported_architectures/opt.py
+++ b/transformer_lens/model_bridge/supported_architectures/opt.py
@@ -11,6 +11,7 @@
     AttentionBridge,
     BlockBridge,
     EmbeddingBridge,
+    LinearBridge,
     MLPBridge,
     NormalizationBridge,
     PosEmbedBridge,
@@ -72,7 +73,13 @@ def __init__(self, cfg: Any) -> None:
                     "ln1": NormalizationBridge(name="self_attn_layer_norm", config=self.cfg),
                     "attn": AttentionBridge(name="self_attn", config=self.cfg),
                     "ln2": NormalizationBridge(name="final_layer_norm", config=self.cfg),
-                    "mlp": MLPBridge(name="mlp"),
+                    "mlp": MLPBridge(
+                        name=None,  # No MLP container; fc1/fc2 are on block
+                        submodules={
+                            "in": LinearBridge(name="fc1"),
+                            "out": LinearBridge(name="fc2"),
+                        },
+                    ),
                 },
             ),
             "ln_final": NormalizationBridge(name="model.decoder.final_layer_norm", config=self.cfg),
diff --git a/transformer_lens/utilities/bridge_components.py b/transformer_lens/utilities/bridge_components.py
index 396990121..c3d03c1c6 100644
--- a/transformer_lens/utilities/bridge_components.py
+++ b/transformer_lens/utilities/bridge_components.py
@@ -26,7 +26,9 @@ def collect_all_submodules_of_component(
         Dictionary mapping submodule names to their respective submodules
     """
     for component_submodule in component.submodules.values():
-        submodules[block_prefix + component_submodule.name] = component_submodule
+        # Skip components without names (e.g., OPT's MLP container)
+        if component_submodule.name is not None:
+            submodules[block_prefix + component_submodule.name] = component_submodule
 
         # If the component is a list item, we need to collect all submodules of the block bridge
         if component_submodule.is_list_item:
@@ -53,6 +55,8 @@ def collect_components_of_block_bridge(
     """
 
     # Retrieve the remote component list from the adapter (we need a ModuleList to iterate over)
+    if component.name is None:
+        raise ValueError("Block bridge component must have a name")
     remote_module_list = model.adapter.get_remote_component(model.original_model, component.name)
 
     # Make sure the remote component is a ModuleList