Repository navigation
Commit eccf758
Add Whisper HF export (encoder, cross_kv, static-cache decoder) (#23901)
Summary:
Add `examples/audio/export_hf_whisper.py` to export `WhisperForConditionalGeneration` directly through Transformers into one `.ptn` with three methods: `encoder` maps mel features to hidden states, `cross_kv` projects per-layer cross-attention keys then values, and `decoder` consumes token IDs, explicit cache positions, and those projections to produce logits. Self-attention uses zero-initialized mutable static-cache buffers. Positions and the causal mask derive from `cache_position`, so the runner needs no attention-mask input.
`--dump_dir` writes runner inputs and an eager greedy transcript with a fresh decoder, preserving the exported cache buffers. An unspecified forced language token is resolved from the example features through Transformers language detection, retaining subsequent forced-token positions. Sparse forced prefixes are rejected. Generation is capped at `max_target_positions` and skips the unused decoder call after the last requested token.
The export format is independent of the runtime engine. The exporter and the existing LLM example share lowering reports through `examples/hf_llm/hf_utils.py`. Tests validate native delegation, zero-initialized decoder caches, three-method packaging, and eager numerics. Runtime execution support and runner integration follow in later diffs.
Random-weight loading uses checkpoint generation settings when present, warns when the file is absent, and propagates malformed-config errors. Vector generation supports an unset EOS token by using an empty stop set.
Authored with assistance from Codex.
Reviewed By: digantdesai
Differential Revision: D1252950571 parent 25837ca commit eccf758
7 files changed
Lines changed: 787 additions & 46 deletions
File tree
- backends/native
- examples
- audio
- hf_llm
- test
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
0 commit comments