Skip to content

Add Gemini 3.5 Transcribe as a desktop STT provider - #7137

Merged
ComputelessComputer merged 2 commits into
mainfrom
cursor/gemini-3-5-transcribe-cd6f
Aug 28, 2026
Merged

Add Gemini 3.5 Transcribe as a desktop STT provider#7137
ComputelessComputer merged 2 commits into
mainfrom
cursor/gemini-3-5-transcribe-cd6f

Conversation

@ComputelessComputer

@ComputelessComputer ComputelessComputer commented Aug 28, 2026

Copy link
Copy Markdown
Collaborator

Summary

Problem: Gemini 3.5 Transcribe is now available in Google AI Studio, with a live model for captions during recording and a file model for speaker labels and word timestamps. Settings only offered Google Cloud Speech-to-Text, which uses a different API, auth, and host.

Fix: Add a Google Gemini STT provider (google_generative_ai) that uses an AI Studio API key. Live captions use gemini-3.5-transcribe-live over the Live API; after recording, gemini-3.5-transcribe goes through the Interactions API with speaker diarization and word timestamps. Selecting the file model forces batch mode so live is not called with the wrong model ID. This stays BYOK and is not added to the Anarlog Pro proxy.

Batch requests omit language hints when none are selected (auto-detect), matching the live path. Live inputTranscription events stay normal committed captions (from_finalize: false) rather than looking like an end-of-stream flush.

Verification

  • cargo test -p owhisper-client --lib
  • cargo test -p owhisper-client --lib adapter::google_generative_ai
  • cargo test -p listener-core --lib
  • cargo test -p listener2-core --lib
  • cargo test -p transcribe-proxy --lib
  • pnpm exec dprint check on changed non-Swift files
  • pnpm -F desktop typecheck
  • pnpm -F desktop test
  • pnpm exec oxlint --quiet --format=github apps/desktop/src/

Skipped cargo test -p tauri-plugin-transcription here because this environment is missing GTK/WebKit for the Tauri/specta export test. plugins/transcription/js/bindings.gen.ts includes google_generative_ai on BatchProvider.

Open in Web Open in Cursor 

Wire Google AI Studio live and file models as a BYOK provider, separate from Google Cloud Speech-to-Text, with live captions during recording and speaker-labeled batch transcription after.

Co-authored-by: John Jeong <ComputelessComputer@users.noreply.github.com>
@netlify

netlify Bot commented Aug 28, 2026

Copy link
Copy Markdown

Deploy Preview for anarlog canceled.

Name Link
🔨 Latest commit 70eba31
🔍 Latest deploy log https://app.netlify.com/projects/anarlog/deploys/6a9137d69cf8720008901d8e

@cla-assistant

cla-assistant Bot commented Aug 28, 2026

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@ComputelessComputer
ComputelessComputer marked this pull request as ready for review August 28, 2026 07:16

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 2 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit e6218b4. Configure here.

Comment thread crates/owhisper-client/src/adapter/google_generative_ai/batch.rs Outdated
duration: (end - start).max(0.0),
is_final,
speech_final: is_final,
from_finalize: is_final,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Live finals marked as finalize events

Medium Severity

Every inputTranscription event sets from_finalize to true along with is_final. Other live adapters keep from_finalize false except for an actual finalize flush. Downstream live handling that special-cases from_finalize can treat each Gemini utterance as an end-of-stream result rather than a normal committed caption.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit e6218b4. Configure here.

Omit language fields on batch when none are selected instead of sending an invalid auto hint, and keep live finals from being marked as finalize flushes.

Co-authored-by: John Jeong <ComputelessComputer@users.noreply.github.com>
@ComputelessComputer
ComputelessComputer merged commit c6fedd7 into main Aug 28, 2026
22 checks passed
cursor Bot pushed a commit that referenced this pull request Aug 28, 2026
Keep the Gemini adapter landed in #7137 and reapply this branch's live from_finalize mapping so only the audioStreamEnd flush is marked as a finalize.

Co-authored-by: John Jeong <ComputelessComputer@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants