MOSS TTS integration with nearly full feature parity (MONOLITH PR WILL NOT MERGE - for reference only - use PRs 501-507 instead)

.pre-commit-config.yaml

@@ -4,7 +4,7 @@ repos:
     hooks:
     -   id: black
 -   repo: https://github.com/pycqa/isort
-    rev: 5.13.2
+    rev: 7.0.0
     hooks:
     -   id: isort
         args:

README.md

@@ -80,6 +80,7 @@ for result in model.generate("Hello from MLX-Audio!", voice="af_heart"):
 |-------|-------------|-----------|------|
 | **Kokoro** | Fast, high-quality multilingual TTS | EN, JA, ZH, FR, ES, IT, PT, HI | [mlx-community/Kokoro-82M-bf16](https://huggingface.co/mlx-community/Kokoro-82M-bf16) |
 | **Qwen3-TTS** | Alibaba's multilingual TTS with voice design | ZH, EN, JA, KO, + more | [mlx-community/Qwen3-TTS-12Hz-1.7B-VoiceDesign-bf16](https://huggingface.co/mlx-community/Qwen3-TTS-12Hz-1.7B-VoiceDesign-bf16) |
+| **MOSS-TTS Family** | Delay/Local/TTSD/VoiceGenerator/SoundEffect/Realtime | Multilingual + pinyin/ipa input modes | [OpenMOSS-Team/MOSS-TTS](https://huggingface.co/OpenMOSS-Team/MOSS-TTS) |
 | **CSM** | Conversational Speech Model with voice cloning | EN | [mlx-community/csm-1b](https://huggingface.co/mlx-community/csm-1b) |
 | **Dia** | Dialogue-focused TTS | EN | [mlx-community/Dia-1.6B-fp16](https://huggingface.co/mlx-community/Dia-1.6B-fp16) |
 | **OuteTTS** | Efficient TTS model | EN | [mlx-community/OuteTTS-1.0-0.6B-fp16](https://huggingface.co/mlx-community/OuteTTS-1.0-0.6B-fp16) |
@@ -173,6 +174,16 @@ audio = results[0].audio  # mx.array
 
 See the [Qwen3-TTS README](mlx_audio/tts/models/qwen3_tts/README.md) for voice cloning, CustomVoice, VoiceDesign, and all available models.
 
+### MOSS-TTS Family
+
+OpenMOSS MOSS-TTS variants are available through the unified MOSS runtime (`moss_tts` and `moss_tts_realtime` model packages), including Delay, Local, TTSD, VoiceGenerator, SoundEffect, and Realtime.
+
+See the [MOSS-TTS README](mlx_audio/tts/models/moss_tts/README.md) for presets, CLI recipes, and parity artifacts.
+Pronunciation-control guidance (tone-numbered pinyin, `/.../`-wrapped IPA, and optional helper conversion flows) is covered there and in [`examples/moss_tts_pronunciation_control.py`](examples/moss_tts_pronunciation_control.py).
+Phase 7 showcase scripts include [`examples/moss_tts_continuation_showcase.py`](examples/moss_tts_continuation_showcase.py), [`examples/moss_tts_realtime_multiturn_agent.py`](examples/moss_tts_realtime_multiturn_agent.py), and [`examples/moss_tts_showcase_album.py`](examples/moss_tts_showcase_album.py).
+For realtime session controls (voice-prompt API, prompt packing, repetition-window sampling), see [MOSS-TTS-Realtime README](mlx_audio/tts/models/moss_tts_realtime/README.md).
+For architecture details and runtime contracts, see [`TECHNICAL_OVERVIEW.md`](mlx_audio/tts/models/moss_tts/TECHNICAL_OVERVIEW.md) and [`moss_tts_realtime/TECHNICAL_OVERVIEW.md`](mlx_audio/tts/models/moss_tts_realtime/TECHNICAL_OVERVIEW.md).
+
 ### CSM (Voice Cloning)
 
 Clone any voice using a reference audio sample:

examples/moss_sound_effects.py

@@ -0,0 +1,97 @@
+#!/usr/bin/env python
+"""Sound-effect generation with MOSS-SoundEffect.
+
+This example demonstrates ambient and event-conditioned generation for the
+SoundEffect variant.
+
+Usage:
+    uv run python examples/moss_sound_effects.py
+    uv run python examples/moss_sound_effects.py --ambient-sound "busy cafe" --sound-event "cups clinking"
+"""
+
+from __future__ import annotations
+
+import argparse
+
+from mlx_audio.tts.generate import generate_audio
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser(description="MOSS-SoundEffect example")
+    parser.add_argument(
+        "--model",
+        default="OpenMOSS-Team/MOSS-SoundEffect",
+        help="SoundEffect model id or local path",
+    )
+    parser.add_argument(
+        "--ambient-sound",
+        default="Thunder and rain over a city street.",
+        help="Ambient scene description",
+    )
+    parser.add_argument(
+        "--sound-event",
+        default="storm",
+        help="Primary sound event cue",
+    )
+    parser.add_argument(
+        "--quality",
+        default="high",
+        help="Quality hint",
+    )
+    parser.add_argument(
+        "--preset",
+        default="soundeffect",
+        help="Sampling preset",
+    )
+    parser.add_argument(
+        "--tokens",
+        type=int,
+        default=140,
+        help="Target token budget",
+    )
+    parser.add_argument(
+        "--max-tokens",
+        type=int,
+        default=260,
+        help="Safety cap for generation",
+    )
+    parser.add_argument(
+        "--output-dir",
+        default="outputs/moss_sound_effects",
+        help="Directory for generated outputs",
+    )
+    parser.add_argument(
+        "--file-prefix",
+        default="sound_effect",
+        help="Output filename prefix",
+    )
+    parser.add_argument(
+        "--verbose",
+        action="store_true",
+        help="Print runtime stats",
+    )
+    return parser.parse_args()
+
+
+def main() -> None:
+    args = parse_args()
+    # `text` is intentionally omitted: generate_audio() mirrors CLI behavior and
+    # backfills `text` from `ambient_sound` when needed for SoundEffect prompts.
+    generate_audio(
+        text=None,
+        model=args.model,
+        preset=args.preset,
+        ambient_sound=args.ambient_sound,
+        sound_event=args.sound_event,
+        quality=args.quality,
+        tokens=args.tokens,
+        max_tokens=args.max_tokens,
+        output_path=args.output_dir,
+        file_prefix=args.file_prefix,
+        verbose=args.verbose,
+        play=False,
+    )
+
+
+if __name__ == "__main__":
+    main()

examples/moss_tts_basic.py

@@ -0,0 +1,141 @@
+#!/usr/bin/env python
+"""Basic MOSS-TTS usage for Local/Delay checkpoints.
+
+This example demonstrates:
+- canonical model loading through `mlx_audio.tts.generate.generate_audio`
+- variant presets (`moss_tts_local`, `moss_tts`)
+- `tokens` / `duration_s` control
+- input representation (`text`, `pinyin`, `ipa`)
+- optional streaming output
+- Local-only `n_vq_for_inference` override
+
+Usage:
+    uv run python examples/moss_tts_basic.py
+    uv run python examples/moss_tts_basic.py --model OpenMOSS-Team/MOSS-TTS --preset moss_tts
+    uv run python examples/moss_tts_basic.py --input-type pinyin --text "ni hao, huan ying shi yong MOSS"
+    uv run python examples/moss_tts_basic.py --stream --streaming-interval 0.5
+"""
+
+from __future__ import annotations
+
+import argparse
+
+from mlx_audio.tts.generate import generate_audio
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser(description="Basic MOSS-TTS generation example")
+    parser.add_argument(
+        "--model",
+        default="OpenMOSS-Team/MOSS-TTS-Local-Transformer",
+        help="MOSS model id or local path",
+    )
+    parser.add_argument(
+        "--text",
+        default="Hello from the MOSS-TTS MLX runtime.",
+        help="Input text",
+    )
+    parser.add_argument(
+        "--preset",
+        default="moss_tts_local",
+        help="Sampling preset (e.g. moss_tts_local, moss_tts)",
+    )
+    parser.add_argument(
+        "--input-type",
+        choices=["text", "pinyin", "ipa"],
+        default="text",
+        help="Input representation for supported models",
+    )
+    parser.add_argument(
+        "--language",
+        default=None,
+        help="Optional language hint",
+    )
+    parser.add_argument(
+        "--instruct",
+        default=None,
+        help="Optional instruction/style hint",
+    )
+    parser.add_argument(
+        "--quality",
+        default="balanced",
+        help="Quality hint (draft/balanced/high/max/custom:...)",
+    )
+    parser.add_argument(
+        "--tokens",
+        type=int,
+        default=None,
+        help="Target token budget (takes precedence over --duration-s)",
+    )
+    parser.add_argument(
+        "--duration-s",
+        type=float,
+        default=8.0,
+        help="Duration hint in seconds when --tokens is not provided",
+    )
+    parser.add_argument(
+        "--max-tokens",
+        type=int,
+        default=240,
+        help="Safety cap for generation loop",
+    )
+    parser.add_argument(
+        "--n-vq-for-inference",
+        type=int,
+        default=None,
+        help="Local-only depth override (1..n_vq)",
+    )
+    parser.add_argument(
+        "--stream",
+        action="store_true",
+        help="Emit and save stream chunks instead of one merged clip",
+    )
+    parser.add_argument(
+        "--streaming-interval",
+        type=float,
+        default=0.5,
+        help="Chunk interval in seconds for streaming mode",
+    )
+    parser.add_argument(
+        "--output-dir",
+        default="outputs/moss_tts_basic",
+        help="Directory for generated audio",
+    )
+    parser.add_argument(
+        "--file-prefix",
+        default="basic",
+        help="Output filename prefix",
+    )
+    parser.add_argument(
+        "--verbose",
+        action="store_true",
+        help="Print runtime stats for each generated segment",
+    )
+    return parser.parse_args()
+
+
+def main() -> None:
+    args = parse_args()
+    generate_audio(
+        text=args.text,
+        model=args.model,
+        preset=args.preset,
+        input_type=args.input_type,
+        language=args.language,
+        instruct=args.instruct,
+        quality=args.quality,
+        tokens=args.tokens,
+        duration_s=args.duration_s,
+        max_tokens=args.max_tokens,
+        n_vq_for_inference=args.n_vq_for_inference,
+        stream=args.stream,
+        streaming_interval=args.streaming_interval,
+        output_path=args.output_dir,
+        file_prefix=args.file_prefix,
+        verbose=args.verbose,
+        play=False,
+    )
+
+
+if __name__ == "__main__":
+    main()