Add Nemotron 3.5 ASR Streaming#46565
Draft
eustlb wants to merge 117 commits into
Draft
Conversation
Contributor
|
Thank you for your contribution 🤗! CI Security Gate — automatic approval blockedThis PR was not automatically approved for CI because the security gate failed. Possible reasons:
See the workflow run for the exact violations. A maintainer can review and manually approve CI if a finding is a false positive. |
…n mixin The base model was renamed nemotron_asr -> nemotron_asr_streaming (NemotronAsr* -> NemotronAsrStreaming*) and evolved (Parakeet-based RNN-T generation, refactored encoder subsampling, stateful processor num_lookahead API). Repoint Nemotron3_5Asr onto it: - generation: Nemotron3_5AsrGenerationMixin now subclasses NemotronAsrStreamingGenerationMixin and only overrides generate() to stash prompt_ids (the offline encode and every streaming chunk read it via get_audio_features). ~370 lines removed. - config/modeling/processing/feature-extraction: inherit the NemotronAsrStreaming* classes; forward returns the streaming cache fields (encoder_past_key_values, padding_cache). - processor __call__ re-based on the new API (default_num_lookahead_tokens via set_num_lookahead_tokens; dropped streaming_latency_ms). - conversion: updated encoder subsampling weight mapping (conv_in + depthwise/pointwise). - re-added auto registrations (wiped by the rename); encoder reuses NemotronAsrStreamingEncoder.
Contributor
|
[For maintainers] Suggested jobs to run (before merge) run-slow: auto, nemotron3_5_asr |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?