Add Gemini 3.5 Transcribe as a desktop STT provider - #7137
Conversation
Wire Google AI Studio live and file models as a BYOK provider, separate from Google Cloud Speech-to-Text, with live captions during recording and speaker-labeled batch transcription after. Co-authored-by: John Jeong <ComputelessComputer@users.noreply.github.com>
✅ Deploy Preview for anarlog canceled.
|
|
|
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 2 potential issues.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit e6218b4. Configure here.
| duration: (end - start).max(0.0), | ||
| is_final, | ||
| speech_final: is_final, | ||
| from_finalize: is_final, |
There was a problem hiding this comment.
Live finals marked as finalize events
Medium Severity
Every inputTranscription event sets from_finalize to true along with is_final. Other live adapters keep from_finalize false except for an actual finalize flush. Downstream live handling that special-cases from_finalize can treat each Gemini utterance as an end-of-stream result rather than a normal committed caption.
Reviewed by Cursor Bugbot for commit e6218b4. Configure here.
Omit language fields on batch when none are selected instead of sending an invalid auto hint, and keep live finals from being marked as finalize flushes. Co-authored-by: John Jeong <ComputelessComputer@users.noreply.github.com>
Keep the Gemini adapter landed in #7137 and reapply this branch's live from_finalize mapping so only the audioStreamEnd flush is marked as a finalize. Co-authored-by: John Jeong <ComputelessComputer@users.noreply.github.com>


Summary
Problem: Gemini 3.5 Transcribe is now available in Google AI Studio, with a live model for captions during recording and a file model for speaker labels and word timestamps. Settings only offered Google Cloud Speech-to-Text, which uses a different API, auth, and host.
Fix: Add a Google Gemini STT provider (
google_generative_ai) that uses an AI Studio API key. Live captions usegemini-3.5-transcribe-liveover the Live API; after recording,gemini-3.5-transcribegoes through the Interactions API with speaker diarization and word timestamps. Selecting the file model forces batch mode so live is not called with the wrong model ID. This stays BYOK and is not added to the Anarlog Pro proxy.Batch requests omit language hints when none are selected (auto-detect), matching the live path. Live
inputTranscriptionevents stay normal committed captions (from_finalize: false) rather than looking like an end-of-stream flush.Verification
cargo test -p owhisper-client --libcargo test -p owhisper-client --lib adapter::google_generative_aicargo test -p listener-core --libcargo test -p listener2-core --libcargo test -p transcribe-proxy --libpnpm exec dprint checkon changed non-Swift filespnpm -F desktop typecheckpnpm -F desktop testpnpm exec oxlint --quiet --format=github apps/desktop/src/Skipped
cargo test -p tauri-plugin-transcriptionhere because this environment is missing GTK/WebKit for the Tauri/specta export test.plugins/transcription/js/bindings.gen.tsincludesgoogle_generative_aionBatchProvider.