Skip to content

PoR: Remove unused hf_pretrained compatibility APIs #4717

Description

@yaoyu-33

Summary

Clean up unused and misleading compatibility APIs under
src/megatron/bridge/models/hf_pretrained.

A repository-wide usage audit at commit
030500e291cff2f4cb739453c93c8b9b0427890f found two dead symbols, a VLM
wrapper that is never instantiated by production code, and several custom
StateDict conveniences used only by tests and documentation.

Scope

Remove dead symbols

  • Remove the unused SAFE_REPOS constant from hf_pretrained/utils.py.
  • Remove stale documentation that still describes organization-based implicit
    trust. trust_remote_code must remain an explicit user decision.
  • Remove the unreachable private
    SafeTensorsStateSource._get_key_to_filename_map() pass-through. The live
    property already calls _cached_get_key_to_filename_map() directly.

Remove PreTrainedVLM

  • Remove hf_pretrained/vlm.py and the root package export.
  • Replace the 15 VLM bridge imports and provider_bridge annotations with the
    actual wrapper type passed by AutoBridge, currently
    PreTrainedCausalLM, or a smaller existing structural type if that better
    reflects the consumed contract.
  • Update mocks, unit tests, model-support skills, and documentation that refer
    to PreTrainedVLM.
  • Add focused coverage proving the wrapper type and required attributes passed
    through AutoBridge.from_hf_pretrained(...).to_megatron_provider() for a VLM
    architecture.
  • Preserve the transformers_compat import side effect currently reached
    through the remaining wrapper/import path.

Simplify StateDict

  • Remove StateDict.regex() and StateDict.glob().
  • Remove the explicit StateDict.items() override. Standard
    collections.abc.Mapping.items() behavior should remain available through
    inheritance.
  • Update tests and documentation to use StateDict.__getitem__ pattern access,
    iteration, or standard Mapping methods as appropriate.
  • Do not remove production-used exact lookup, has_glob, keys, get,
    __iter__, __len__, state sources, or save/load paths.

Documentation correction

  • Replace the package README's nonexistent from_hf_pretrained() examples
    with the implemented from_pretrained() constructor where those examples
    remain relevant after the cleanup.

Validation

  • No production, test, documentation, example, or skill references to
    PreTrainedVLM, SAFE_REPOS, StateDict.regex, or StateDict.glob
    remain.
  • The custom StateDict.items implementation is gone and inherited
    Mapping.items() behavior is covered.
  • AutoBridge VLM conversion/provider construction retains focused type-flow
    coverage.
  • Security tests continue to require explicit trust_remote_code opt-in.
  • Focused hf_pretrained, state, AutoBridge, and affected VLM bridge tests
    pass.
  • uv run pre-commit run --all-files passes.

Implementation guidance

Keep this as a behavior-preserving cleanup for active conversion paths. The
removed public compatibility surfaces are intentionally accepted as cleanup
because there are no in-repository runtime consumers. Review the final diff for
external-facing release-note implications before merge.

Metadata

Metadata

Assignees

No one assigned

    Labels

    PoRPlan of record item for roadmap and release trackingarea:modelModel implementations and HF bridge logicbreaking-changePublic behavior or API compatibility changesfeatureNew capabilities, enhancements, or enablement worktrackingTracking issue for an ongoing project with smaller steps

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions