Audio: add NeMo FastConformer ASR adapter by mohammadaaftabv · Pull Request #2254 · NVIDIA-NeMo/Curator

mohammadaaftabv · 2026-07-27T05:59:24Z

Summary

Adds a NeMo ASR adapter for the shared ASRStage / ASRAdapter contract and migrates Curator's in-repository NeMo ASR consumers to that adapter-based path.

The resulting ownership split is:

ASRStage owns file decoding, mono conversion, resampling, language filtering, batching, skip handling, output assembly, resources, and lifecycle routing.
NeMoASRAdapter owns NeMo checkpoint download, worker-local model load/unload, optional FastConformer local-attention conversion, optional RNNT CUDA-graph decoder configuration, batched transcription, and NeMo output normalization.

Changes

Add NeMoASRAdapter behind the shared ASR adapter protocol.
Keep model-specific options in adapter_kwargs rather than forcing them onto every ASR adapter.
Migrate the FLEURS benchmark, YAML, README, and notebook to ASRStage + NeMoASRAdapter.
Add a minimal shared-runner FastConformer tutorial using README.md + pipeline.yaml.
Update the shared YAML runner so pipeline is the default Hydra config name.
Keep Qwen-Omni model revision adapter-owned while preserving its existing pipeline behavior.
Move TorchCodec into the audio-common dependency path and select its CPU wheel for audio installs while retaining the Torch 2.11-compatible version override already present on main.
Update focused unit, configuration, lazy-import, stage, tagging, and real-GPU smoke coverage.

Intentional migration

This PR removes the legacy nemo_curator.stages.audio.inference.asr.asr_nemo.InferenceAsrNemoStage module and migrates all in-repository consumers to the shared stage/adapter path. It does not add a compatibility re-export; downstream users of the legacy import must move to:

ASRStage(
    adapter_target="nemo_curator.models.asr.nemo_asr.NeMoASRAdapter",
    model_id="nvidia/stt_en_fastconformer_ctc_large",
)

Review feedback addressed

Removed redundant adapter type/value checks.
Moved torch and OmegaConf imports to module scope while retaining the optional NeMo lazy import.
Consolidated the GPU smoke test into tests/models/asr/test_nemo_asr.py.
Made --config-name pipeline optional for shared-runner tutorials and added CLI coverage.
Removed the exact direct TorchCodec pin and documented the ABI-compatible shared override.

Validation after rebasing onto current `main`

126 focused CPU tests passed; the marked real-GPU smoke test was deselected in this CPU test command.
Ruff lint passed.
CI-equivalent pre-commit checks passed.
uv lock --check passed with uv 0.12.2.
git diff --check passed.

copy-pr-bot · 2026-07-27T05:59:27Z

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

github-actions · 2026-08-04T08:05:18Z

🌿 Preview your docs: https://nvidia-preview-aaftabv-audio-nemo-fastconformer-adapter.docs.buildwithfern.com/nemo/curator

Here are the markdown pages you've updated:

greptile-apps · 2026-08-04T18:40:06Z

Greptile Summary

The PR introduces a NeMo FastConformer adapter and migrates in-repository NeMo ASR usage to the shared ASRStage lifecycle.

Adds NeMoASRAdapter with checkpoint staging, worker-local model lifecycle, FastConformer local-attention conversion, RNNT decoder configuration, and normalized batched results.
Migrates benchmarks, tutorials, configuration, and tests from InferenceAsrNemoStage to ASRStage.
Makes pipeline.yaml the default Hydra configuration and updates shared audio dependency resolution.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains; the previously reported legacy-import concern is addressed by the explicitly intentional breaking migration and accompanying guidance.

Important Files Changed

Filename	Overview
nemo_curator/models/asr/nemo_asr.py	Adds the NeMo ASR adapter, including model download/load lifecycle, optional decoder configuration, batched transcription, and output normalization.
nemo_curator/stages/audio/inference/asr/stage.py	Updates the shared ASR stage to construct adapters with adapter-owned options and route NeMo inference through the common stage lifecycle.
nemo_curator/stages/audio/inference/asr/asr_nemo.py	Removes the legacy NeMo-specific stage as part of the explicitly documented adapter migration.
nemo_curator/config/run.py	Makes pipeline.yaml the default Hydra configuration while retaining explicit config-name support.
pyproject.toml	Moves TorchCodec into the shared audio dependency path and configures the appropriate CPU package source.
tutorials/audio/nemo_fastconformer/pipeline.yaml	Adds a shared-runner FastConformer pipeline using ASRStage and NeMoASRAdapter.

Sequence Diagram

sequenceDiagram
    participant Runner as Pipeline Runner
    participant Stage as ASRStage
    participant Adapter as NeMoASRAdapter
    participant NeMo as NeMo ASRModel
    Runner->>Stage: setup_on_node()
    Stage->>Adapter: download_weights_on_node()
    Runner->>Stage: setup()
    Stage->>Adapter: load_model(num_gpus)
    Runner->>Stage: process_batch(AudioTask[])
    Stage->>Stage: decode, mono-convert, resample, filter
    Stage->>Adapter: transcribe_batch(items)
    Adapter->>NeMo: transcribe(waveforms)
    NeMo-->>Adapter: hypotheses/text
    Adapter-->>Stage: ASRResult[]
    Stage-->>Runner: updated AudioTask[]
    Runner->>Stage: teardown()
    Stage->>Adapter: unload_model()

_{Reviews (12): Last reviewed commit: "Fix ASR adapter consistency after rebase" | Re-trigger Greptile}

sarahyurick · 2026-08-05T19:19:38Z

+    _model: Any = field(default=None, init=False, repr=False)
+
+    def __post_init__(self) -> None:
+        if not self.model_id:


Some of these checks are overkill IMO. Like this and the check for whether enable_local_attention (and others) is a boolean.

sarahyurick · 2026-08-05T19:20:01Z

+            msg = f"NeMoASRAdapter requires num_gpus to be 0 or 1, got {num_gpus!r}"
+            raise ValueError(msg)
+
+        import torch


Let's use top-level imports for most of these.

sarahyurick · 2026-08-05T19:20:53Z

Let's move all the tests to test_nemo_asr.

sarahyurick · 2026-08-05T19:22:43Z

+```bash
+python nemo_curator/config/run.py \
+  --config-path ../../tutorials/audio/nemo_fastconformer \
+  --config-name pipeline \


Can config-name be optional?

sarahyurick · 2026-08-05T19:24:13Z

    "nemo_toolkit[asr]>=2.7.2; platform_machine == 'x86_64' and platform_system != 'Darwin'",
    "soundfile>=0.12.0",
    "torchaudio",
+    "torchcodec==0.11.1; platform_machine == 'x86_64' and platform_system != 'Darwin'",


Why the pin?

sarahyurick · 2026-08-07T17:50:14Z

    }
   },
   "outputs": [],
   "source": [
-    "configure_loguru()\n",
-    "ray_client.stop()"
+    "configure_loguru()\nray_client.stop()"


Just wondering why do we need configure_loguru here?

sarahyurick · 2026-08-07T17:51:22Z

 explicit = true

+[[tool.uv.index]]
+name = "pytorch-cpu"


I was also wondering why it needs this?

mohammadaaftabv · 2026-08-10T09:20:09Z

/ok to test 9e38391

Signed-off-by: aaftaabv@gmail.com <aaftaabv@gmail.com>

mohammadaaftabv · 2026-08-11T09:14:32Z

/ok to test 9a7ef2c

Signed-off-by: aaftaabv@gmail.com <aaftaabv@gmail.com>

mohammadaaftabv · 2026-08-11T09:46:34Z

/ok to test 16a0c81

This was referenced Jul 27, 2026

Audio: add reusable ASR model-input segmentation #2255

Closed

Audio: add Qwen3-ASR adapter #2257

Merged

mohammadaaftabv force-pushed the aaftabv/audio-nemo-fastconformer-adapter branch from 6e97d31 to 42b7cb6 Compare July 28, 2026 10:14

mohammadaaftabv mentioned this pull request Jul 28, 2026

Audio: support local NeMo ASR checkpoints #2272

Draft

mohammadaaftabv force-pushed the aaftabv/audio-nemo-fastconformer-adapter branch from c8e6c58 to a54f655 Compare August 4, 2026 08:02

mohammadaaftabv marked this pull request as ready for review August 4, 2026 18:36

mohammadaaftabv requested review from a team as code owners August 4, 2026 18:36

mohammadaaftabv requested review from ayushdg and removed request for a team August 4, 2026 18:36

mohammadaaftabv force-pushed the aaftabv/audio-nemo-fastconformer-adapter branch 2 times, most recently from a890d75 to e6329f8 Compare August 5, 2026 17:34

sarahyurick reviewed Aug 5, 2026

View reviewed changes

mohammadaaftabv force-pushed the aaftabv/audio-nemo-fastconformer-adapter branch from c414de6 to 9e73313 Compare August 6, 2026 19:16

mohammadaaftabv requested a review from sarahyurick August 6, 2026 19:24

sarahyurick reviewed Aug 7, 2026

View reviewed changes

mohammadaaftabv force-pushed the aaftabv/audio-nemo-fastconformer-adapter branch from a395afd to 9e38391 Compare August 7, 2026 18:18

mohammadaaftabv requested a review from sarahyurick August 7, 2026 18:20

copy-pr-bot Bot temporarily deployed to public August 10, 2026 09:20 Inactive

copy-pr-bot Bot temporarily deployed to test August 10, 2026 09:20 Inactive

copy-pr-bot Bot temporarily deployed to nemo-ci August 10, 2026 09:20 Inactive

copy-pr-bot Bot temporarily deployed to nemo-ci August 11, 2026 08:59 Inactive

copy-pr-bot Bot temporarily deployed to public August 11, 2026 09:04 Inactive

mohammadaaftabv added 19 commits August 11, 2026 14:35

Audio: isolate NeMo adapter sizing diff

54f8a81

Signed-off-by: aaftaabv@gmail.com <aaftaabv@gmail.com>

Audio: build NeMo adapter on shared ASR contract

524715e

Signed-off-by: aaftaabv@gmail.com <aaftaabv@gmail.com>

Audio: reuse NeMo transcription normalization

afe16b1

Signed-off-by: aaftaabv@gmail.com <aaftaabv@gmail.com>

Audio: keep FastConformer integration adapter-only

7721f91

Signed-off-by: aaftaabv@gmail.com <aaftaabv@gmail.com>

Audio: migrate FastConformer users to ASR adapter

5813b6a

Signed-off-by: aaftaabv@gmail.com <aaftaabv@gmail.com>

Docs: remove Fern changes

b5c661b

Signed-off-by: aaftaabv@gmail.com <aaftaabv@gmail.com>

Audio: avoid TorchCodec for file I/O

33b7c0a

Signed-off-by: aaftaabv@gmail.com <aaftaabv@gmail.com>

Remove Claude changes from audio adapter PR

fcc4d80

Signed-off-by: aaftaabv@gmail.com <aaftaabv@gmail.com>

Audio: use CPU TorchCodec with CUDA PyTorch

200e211

Signed-off-by: aaftaabv@gmail.com <aaftaabv@gmail.com>

Audio: scope TorchCodec CPU source to audio extra

1517fe6

Signed-off-by: aaftaabv@gmail.com <aaftaabv@gmail.com>

Audio: keep model options adapter-owned

8ea5de8

Signed-off-by: aaftaabv@gmail.com <aaftaabv@gmail.com>

Audio: refresh lock for current main dependency stack

18f7213

Signed-off-by: aaftaabv@gmail.com <aaftaabv@gmail.com>

Audio: make FLEURS notebook CUDA-portable

cc22cc6

Signed-off-by: aaftaabv@gmail.com <aaftaabv@gmail.com>

CI: refresh FLEURS notebook secret baseline

d5a7816

Signed-off-by: aaftaabv@gmail.com <aaftaabv@gmail.com>

Address FastConformer review feedback

42aa2ac

Signed-off-by: aaftaabv@gmail.com <aaftaabv@gmail.com>

Address follow-up FastConformer review feedback

0048fd6

Signed-off-by: aaftaabv@gmail.com <aaftaabv@gmail.com>

Refresh FLEURS tutorial outputs after review fixes

08df983

Signed-off-by: aaftaabv@gmail.com <aaftaabv@gmail.com>

CI: refresh FLEURS notebook secret baseline

2c7fae0

Signed-off-by: aaftaabv@gmail.com <aaftaabv@gmail.com>

Address NeMo adapter constant ownership review

9a7ef2c

Signed-off-by: aaftaabv@gmail.com <aaftaabv@gmail.com>

Fix ASR adapter consistency after rebase

16a0c81

Signed-off-by: aaftaabv@gmail.com <aaftaabv@gmail.com>

Navigation Menu

Uh oh!

Audio: add NeMo FastConformer ASR adapter - #2254

Audio: add NeMo FastConformer ASR adapter#2254
mohammadaaftabv merged 20 commits into
NVIDIA-NeMo:mainfrom
mohammadaaftabv:aaftabv/audio-nemo-fastconformer-adapter

mohammadaaftabv commented Jul 27, 2026 •

edited

Loading

Uh oh!

copy-pr-bot Bot commented Jul 27, 2026

Uh oh!

github-actions Bot commented Aug 4, 2026

Uh oh!

greptile-apps Bot commented Aug 4, 2026 •

edited

Loading

Uh oh!

sarahyurick Aug 5, 2026

Uh oh!

sarahyurick Aug 5, 2026

Uh oh!

sarahyurick Aug 5, 2026

Uh oh!

sarahyurick Aug 5, 2026

Uh oh!

sarahyurick Aug 5, 2026

Uh oh!

sarahyurick Aug 7, 2026

Uh oh!

sarahyurick Aug 7, 2026

Uh oh!

mohammadaaftabv commented Aug 10, 2026

Uh oh!

mohammadaaftabv commented Aug 11, 2026

Uh oh!

mohammadaaftabv commented Aug 11, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

mohammadaaftabv commented Jul 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

Intentional migration

Review feedback addressed

Validation after rebasing onto current main

Uh oh!

copy-pr-bot Bot commented Jul 27, 2026

Uh oh!

github-actions Bot commented Aug 4, 2026

Uh oh!

greptile-apps Bot commented Aug 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Summary

Confidence Score: 5/5

Important Files Changed

Sequence Diagram

Uh oh!

sarahyurick Aug 5, 2026

Choose a reason for hiding this comment

Uh oh!

sarahyurick Aug 5, 2026

Choose a reason for hiding this comment

Uh oh!

sarahyurick Aug 5, 2026

Choose a reason for hiding this comment

Uh oh!

sarahyurick Aug 5, 2026

Choose a reason for hiding this comment

Uh oh!

sarahyurick Aug 5, 2026

Choose a reason for hiding this comment

Uh oh!

sarahyurick Aug 7, 2026

Choose a reason for hiding this comment

Uh oh!

sarahyurick Aug 7, 2026

Choose a reason for hiding this comment

Uh oh!

mohammadaaftabv commented Aug 10, 2026

Uh oh!

mohammadaaftabv commented Aug 11, 2026

Uh oh!

mohammadaaftabv commented Aug 11, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

mohammadaaftabv commented Jul 27, 2026 •

edited

Loading

Validation after rebasing onto current `main`

greptile-apps Bot commented Aug 4, 2026 •

edited

Loading