Add GPT Transcribe support

docs/CONFIGURATION.md

@@ -611,57 +611,66 @@ Connect to any backend, local or cloud, via your own custom configuration:
 
 ### Realtime WebSocket
 
-Low-latency streaming transcription.
+Persistent WebSocket streaming. `realtime_mode` selects `transcribe` (speech-to-text, the default) or `converse`
+(voice-to-AI), but converse support varies by provider:
 
-#### OpenAI Realtime
+| Provider | `transcribe` | `converse` |
+| --- | --- | --- |
+| OpenAI | Yes | Yes, on a `gpt-realtime-*` model |
+| Google Gemini | Yes | Yes |
+| ElevenLabs | Yes | No — hyprwhspr ignores `realtime_mode` |
+| Custom | Yes | Yes |
 
-Two modes available (set `realtime_mode` in your config):
+Custom endpoints speak the OpenAI Realtime protocol. Set `websocket_provider: "custom"` and `websocket_url`.
 
-- **transcribe** (default) - Pure speech-to-text, more expensive than HTTP
-- **converse** - Voice-to-AI: speak and get AI responses, configurable via `hyprwhspr config edit`
+#### OpenAI Realtime
 
-Available transcription models:
+Dedicated transcription models, all requiring `realtime_mode: "transcribe"`:
 
-- **GPT Live Transcribe** (`gpt-live-transcribe`) - Live streaming with the best OSD previews; higher cost
-- **GPT Realtime Whisper** (`gpt-realtime-whisper`) - Legacy streaming transcription model
+| Model | Transcript arrives | Notes |
+| --- | --- | --- |
+| `gpt-transcribe` | After you stop recording | Recommended: accurate, fast, inexpensive |
+| `gpt-live-transcribe` | Live, while you speak | Best OSD previews; higher cost |
+| `gpt-realtime-whisper` | Live, while you speak | Legacy |
 
-Both dedicated transcription models require `realtime_mode: "transcribe"`.
+None of the three support `converse` — that needs a `gpt-realtime-*` model, which `hyprwhspr setup` also offers.
+All three disable server-side VAD and commit the turn when recording stops. `realtime_transcription_delay`
+(partial-result latency vs. accuracy) and the continuous waveform OSD preview apply only to the two live models.
 
 ```jsonc
 {
     "transcription_backend": "realtime-ws",
     "websocket_provider": "openai",
-    "websocket_model": "gpt-live-transcribe",
+    "websocket_model": "gpt-transcribe",
     "realtime_mode": "transcribe",       // "transcribe" or "converse"
-    "realtime_transcription_delay": "low", // "minimal", "low", "medium", "high", or "xhigh"
     "realtime_timeout": 30,              // Advanced: seconds to wait after stop for final transcript
     "realtime_buffer_max_seconds": 5     // Advanced: max unsent audio backlog (seconds) before dropping old chunks
 }
 ```
 
-For `converse` mode, `realtime_conversation_history` defaults to `"session"` so
-the assistant retains prior turns. Set it to `"turn"` to delete each completed
-turn from the provider while reusing the WebSocket; this prevents prior audio
-from being included as conversation context and billed again.
+In `converse` mode, `realtime_conversation_history` controls what the provider retains between turns:
 
-For a stateless voice-to-AI workflow, configure converse mode explicitly:
+- `"turn"` (default) - deletes each completed turn while reusing the WebSocket, so every turn starts with empty
+  context.
+- `"session"` - keeps prior turns for conversational context. The provider re-sends and bills that audio every
+  turn, so cost climbs as the session grows. Choose this only if you want a multi-turn assistant.
+
+For a stateless voice-to-AI workflow:
 
 ```jsonc
 {
     "transcription_backend": "realtime-ws",
     "websocket_provider": "openai",
-    "websocket_model": "gpt-realtime",
+    "websocket_model": "gpt-realtime-2.1",
     "realtime_mode": "converse",
     "realtime_conversation_history": "turn"
 }
 ```
 
-For GPT Live Transcribe, hyprwhspr sends the configured scalar `language` as
-OpenAI's single-entry `languages` hint, streams 24 kHz PCM audio, and explicitly
-commits the turn when recording stops. The delay setting trades partial-result
-latency for transcription accuracy.
-
-Its `prompt` uses the existing active-language fallback: `whisper_prompt_<language>` → `whisper_prompt` → omitted.
+All OpenAI Realtime models stream 24 kHz PCM audio. For GPT Transcribe and GPT Live Transcribe, hyprwhspr sends the
+scalar `language` setting as OpenAI's single-entry `languages` hint and resolves the prompt through
+`whisper_prompt_<language>` → `whisper_prompt` → omitted. For GPT Realtime Whisper it sends a plain scalar
+`language` and no prompt.
 
 #### Google Gemini
 
@@ -678,7 +687,7 @@ Uses native 16kHz audio (no resampling) and server-side VAD.
 {
     "transcription_backend": "realtime-ws",
     "websocket_provider": "google",
-    "websocket_model": "gemini-3.1-flash-live-preview",
+    "websocket_model": "gemini-3.1-flash-live-preview",  // or gemini-2.5-flash-native-audio-preview-12-2025
     "realtime_mode": "transcribe",           // "transcribe" or "converse"
     "realtime_timeout": 30,                  // Advanced: seconds to wait after stop for final transcript
     "realtime_buffer_max_seconds": 5         // Advanced: max unsent audio backlog (seconds) before dropping old chunks

lib/src/backends/realtime_ws_backend.py

@@ -19,21 +19,25 @@
 try:
     from ..backend_utils import normalize_backend
     from ..credential_manager import get_credential
+    from ..openai_realtime_models import (
+        is_continuous,
+        is_transcription_only,
+        uses_language_context,
+    )
     from ..provider_registry import get_provider
 except ImportError:
     from backend_utils import normalize_backend
     from credential_manager import get_credential
+    from openai_realtime_models import (
+        is_continuous,
+        is_transcription_only,
+        uses_language_context,
+    )
     from provider_registry import get_provider
 
 from .base import TranscriptionBackend
 
 
-OPENAI_STREAMING_TRANSCRIPTION_MODELS = frozenset({
-    'gpt-live-transcribe',
-    'gpt-realtime-whisper',
-})
-
-
 class RealtimeWsBackend(TranscriptionBackend):
     """Streaming WebSocket backend; reconnects by full re-initialization on resume."""
 
@@ -67,12 +71,12 @@ def _update_client_language(
         language: Optional[str],
         model_id: Optional[str] = None,
     ) -> None:
-        """Keep GPT Live Transcribe's language hint and prompt in sync."""
+        """Keep new-model transcription language hints and prompts in sync."""
         if not self._realtime_client:
             return
 
         active_model = model_id or getattr(self._realtime_client, 'model', None)
-        if active_model == 'gpt-live-transcribe':
+        if uses_language_context(active_model):
             self._realtime_client.update_transcription_config(
                 language,
                 self._resolve_whisper_prompt(language),
@@ -228,7 +232,7 @@ def _send_direct(audio_chunk: np.ndarray):
             realtime_mode = self.config.get_setting('realtime_mode', 'transcribe')
             if (
                 provider_id == 'openai'
-                and model_id in OPENAI_STREAMING_TRANSCRIPTION_MODELS
+                and is_transcription_only(model_id)
                 and realtime_mode != 'transcribe'
             ):
                 print(
@@ -473,9 +477,9 @@ def _is_partial_preview_enabled(
                 and self.config.get_setting('mic_osd_pill_transcript_enabled', False)
             )
 
-        # Waveform: OpenAI's streaming transcription models emit live deltas.
+        # Waveform: only continuously streaming OpenAI models emit live deltas.
         if provider_id == 'openai':
-            return model_id in OPENAI_STREAMING_TRANSCRIPTION_MODELS
+            return is_continuous(model_id)
 
         return False
 

lib/src/cli/setup.py

@@ -37,13 +37,13 @@
 
 try:
     from ..provider_registry import (
-        PROVIDERS, get_provider, list_providers, get_provider_models,
-        get_model_config, validate_api_key
+        PROVIDERS, get_provider, list_providers, get_model_config,
+        validate_api_key, get_models_for_backend, get_realtime_mode
     )
 except ImportError:
     from provider_registry import (
-        PROVIDERS, get_provider, list_providers, get_provider_models,
-        get_model_config, validate_api_key
+        PROVIDERS, get_provider, list_providers, get_model_config,
+        validate_api_key, get_models_for_backend, get_realtime_mode
     )
 
 try:
@@ -317,17 +317,15 @@ def _prompt_realtime_provider_model_selection():
     print("\n" + "="*60)
     print("Realtime Provider and Model Selection")
     print("="*60)
-    print("\nChoose a realtime streaming provider and model:")
+    print("\nChoose a realtime WebSocket provider and model:")
     print()
 
     realtime_options = []
     for provider_id, provider in PROVIDERS.items():
         if not provider.get('websocket_endpoint'):
             continue
 
-        for model_id, model_data in provider.get('models', {}).items():
-            if not model_data.get('realtime_model', False):
-                continue
+        for model_id, model_data in get_models_for_backend(provider_id, 'realtime-ws').items():
             realtime_options.append((provider_id, provider, model_id, model_data))
             print(
                 f"  [{len(realtime_options)}] "
@@ -446,25 +444,14 @@ def _prompt_remote_provider_selection(filter_realtime: bool = False):
                 continue
 
             # Only show providers that have realtime-capable models
-            models = get_provider_models(provider_id) or {}
-            realtime_model_ids = [
-                model_id
-                for model_id, model_data in models.items()
-                if model_data.get('realtime_model', False)
-            ]
+            realtime_model_ids = list(get_models_for_backend(provider_id, 'realtime-ws'))
             if realtime_model_ids:
                 providers_list.append((provider_id, provider_name, realtime_model_ids))
     else:
-        # REST providers: only include providers with at least one REST-visible model
-        # (i.e. not marked hidden in provider_registry)
+        # REST providers: only include providers offering at least one REST model
         providers_list = []
         for provider_id, provider_name, _model_ids in all_providers_list:
-            models = get_provider_models(provider_id) or {}
-            visible_model_ids = [
-                model_id
-                for model_id, model_data in models.items()
-                if not model_data.get('hidden', False)
-            ]
+            visible_model_ids = list(get_models_for_backend(provider_id, 'rest-api'))
             if visible_model_ids:
                 providers_list.append((provider_id, provider_name, visible_model_ids))
 
@@ -498,19 +485,10 @@ def _prompt_remote_provider_selection(filter_realtime: bool = False):
                 print("="*60)
                 print()
 
-                models = get_provider_models(provider_id)
+                backend = 'realtime-ws' if filter_realtime else 'rest-api'
                 model_list = []
-
-                # Filter models based on backend type
-                for model_id, model_data in models.items():
-                    if filter_realtime:
-                        # Only include realtime models (marked with realtime_model flag)
-                        if not model_data.get('realtime_model', False):
-                            continue
-                    else:
-                        # For REST API, hide models marked as hidden
-                        if model_data.get('hidden', False):
-                            continue
+
+                for model_id, model_data in get_models_for_backend(provider_id, backend).items():
                     model_list.append((model_id, model_data))
                     print(f"  [{len(model_list)}] {model_data['name']} - {model_data['description']}")
 
@@ -956,8 +934,12 @@ def setup_command(python_path: Optional[str] = None):
             log_error(f"Failed to generate realtime configuration: {e}")
             return
 
-        remote_config['realtime_mode'] = 'transcribe'
-        log_info("Realtime mode: transcribe")
+        realtime_mode = get_realtime_mode(provider_id, model_id)
+        remote_config['realtime_mode'] = realtime_mode
+        if realtime_mode == 'converse':
+            log_info("Realtime mode: converse (spoken AI replies)")
+        else:
+            log_info("Realtime mode: transcribe (speech-to-text)")
 
     # Step 1.4: Ensure venv and base dependencies for cloud backends
     if backend_normalized in ['rest-api', 'remote', 'realtime-ws']:

lib/src/config_manager.py

@@ -126,13 +126,13 @@ def __init__(self, verbose: bool = True):
             'rest_audio_format': 'wav',        # Audio format for remote transcription
             # WebSocket realtime backend settings
             'websocket_provider': None,        # Provider identifier for credential lookup (e.g., 'openai', 'google', 'elevenlabs')
-            'websocket_model': None,           # Model identifier (e.g., 'gpt-live-transcribe')
+            'websocket_model': None,           # Model identifier (e.g., 'gpt-transcribe')
             'websocket_url': None,             # Optional: explicit WebSocket URL (auto-derived if None)
             'realtime_timeout': 30,            # Completion timeout (seconds)
             'realtime_buffer_max_seconds': 5,  # Max buffer before dropping chunks
             'realtime_mode': 'transcribe',      # 'transcribe' (speech-to-text) or 'converse' (voice-to-AI)
-            'realtime_transcription_delay': 'low',  # OpenAI streaming delay: minimal|low|medium|high|xhigh
-            'realtime_conversation_history': 'session',  # OpenAI converse mode: session|turn
+            'realtime_transcription_delay': 'low',  # OpenAI continuous transcription delay: minimal|low|medium|high|xhigh
+            'realtime_conversation_history': 'turn',  # OpenAI converse mode: session|turn
             # whisper.cpp (pywhispercpp) backend settings
             'pywhispercpp_use_vad': False,               # Native Silero VAD (strips silence, reduces hallucinations); auto-downloads ~1MB ggml-silero model when enabled
             # ONNX-ASR backend settings (CPU-optimized)

lib/src/openai_realtime_models.py

@@ -0,0 +1,37 @@
+"""Capabilities for OpenAI transcription models used over Realtime WebSocket.
+
+Capabilities live on the model entries in provider_registry so a model is
+described in one place. Unknown models (custom endpoints) report no
+capabilities, which keeps them on server VAD and the singular `language` field.
+"""
+
+try:
+    from .provider_registry import get_realtime_capabilities
+except ImportError:
+    from provider_registry import get_realtime_capabilities
+
+
+def _capability(model_id, name):
+    if not model_id:
+        return False
+    return bool(get_realtime_capabilities('openai', model_id).get(name))
+
+
+def is_transcription_only(model_id) -> bool:
+    """The model rejects realtime_mode="converse"."""
+    return _capability(model_id, 'transcription_only')
+
+
+def uses_manual_commit(model_id) -> bool:
+    """The session disables server VAD and commits the turn on stop."""
+    return _capability(model_id, 'manual_commit')
+
+
+def uses_language_context(model_id) -> bool:
+    """The model takes a `languages` array and an optional `prompt`."""
+    return _capability(model_id, 'language_context')
+
+
+def is_continuous(model_id) -> bool:
+    """The model emits transcript deltas while audio is still arriving."""
+    return _capability(model_id, 'continuous')