Not able to get transcript when sending audio in webm format

[deva-gopalani]

I have got a web app in which I am collecting the audio of users via mic. On collecting the audio I am extracting the base64 value of the raw audio and sending it over a websocket connection to my backend. The code responsible for doing all this is -

navigator.mediaDevices
  .getUserMedia({ audio: true })
  .then((stream) => {
    const url = "<wss-endpoint-to-my-server>"
    const ws = new WebSocket(url)
    
    ws.onopen = () => {
      console.log('WebSocket connected.')
    }

    const options = { mimeType: 'audio/webm;' }
    const mediaRecorder = new MediaRecorder(stream, options)

    mediaRecorder.ondataavailable = (event) => {
      const audioBlob = event.data
      const reader = new FileReader()

      reader.onloadend = () => {
        if (ws.readyState != WebSocket.OPEN) {
          return
        }
        let audioMessage = reader.result
        // extracting base64 string
        audioMessage = audioMessage.split(',')[1]

        // sending to my server
        ws.send(JSON.stringify({ type: 'audio', data: audioMessage }))
      }
      reader.readAsDataURL(audioBlob)
    }
    mediaRecorder.start(200)
  })
  .catch((error) => {
    console.error('Error accessing microphone:', error)
  })

Once I receive this audio in the backend, I decode it and then send it to deepgram for transcription. Now the issue is that I am not able to get any transcription. What could be the possible reason for it? The code present in the backend is as follows -

# Here audio is the base64 string sent from the frontend
data = base64.b64decode(audio)
deepgram_websocket.send(data)

The URL with which I created the websocket connection with deepgram is as follows -
wss://api.deepgram.com/v1/listen?model=nova-2&filler_words=true&diarize=true&language=en&vad_events=true&encoding=opus&sample_rate=48000&channels=1&interim_results=true&utterance_end_ms=1000

I have specified the encoding as opus as the browser is giving me the audio in webm format and the sample rate of it is 48KHz hence the sample rate has been set to the same.

What could be the possible reason for this to not work? If you have any other better recommended approach to make this flow work then also do let me know.

[deepgram-community[bot]]

Thanks for asking your question. Please be sure to reply with as much detail as possible so the community can assist you efficiently.
_{Consider joining our Discord community for more opportunity to engage with your fellow Deepgram users. You can earn points which can be redeemed for cool stuff by being active in our communities!}

[deepgram-community[bot]]

Hey there! It looks like you haven't connected your GitHub account to your Deepgram account. You can do this at https://community.deepgram.com - being verified through this process will allow our team to help you in a much more streamlined fashion.

[deepgram-community[bot]]

It looks like we're missing some important information to help debug your issue. Would you mind providing us with the following details in a reply?

• The deepgram product you are using (e.g Speech to Text, Agent API)
• A request ID that triggered your error or issue.

[deva-gopalani]

@jkroll-deepgram @jpvajda any sort of assistance would be highly appreciated!

[deva-gopalani]

Another thing that I would like to add is that the example stated in this article works perfectly fine with webm audio. It's just that the approach that I am using above is breaking for some reason.

[deva-gopalani]

@jkroll-deepgram @davidvonthenen @jpvajda any kind of assistance would be appreciated!

[jkroll-deepgram]

Hi @deva-gopalani, you mention "I am not able to get any transcription" - can you clarify, are you getting empty transcript responses from Deepgram, or no responses?

Do you have a Deepgram request ID you're able to share? If so, then I can look into the audio to see what Deepgram is receiving.

It's possible that either the audio you're sending to Deepgram is corrupt in some way, or that your audio parameters (e.g. encoding, sample rate) need to be adjusted for the different audio format.

[deva-gopalani]

Hey @jkroll-deepgram, first of all thanks for your reply! To answer your questions - I am not getting any response at all. I am just getting one packet of the type of "Metadata". I just did a session to reproduce the issue. Here is the request ID - feed45ef-b405-4d5c-a9ec-abc94aa358f0.

If the audio is corrupt could you please help me in understanding what am I doing wrong in the above code?

[jkroll-deepgram]

Hi @deva-gopalani, if you are not getting any "Results" type responses from Deepgram, that means that either Deepgram is not receiving audio from you, or you are not listening to the Results event type. I assume that since you have successfully transcribed other audio formats, you are listening to the correct event types, so most likely the issue is related to the audio or how it's being sent.

Can you share the full Metadata message you are receiving? Are any other errors surfaced? I am wondering if it appears that the connection stays open, or if there is an error message associated with the connection closing.

Are you able to save the audio that you're attempting to send to Deepgram, and then review that audio recording to make sure it is present and in the format and duration you expect?

[deva-gopalani]

@jkroll-deepgram here is the compete metadata object -

{'type': 'Metadata', 'transaction_key': 'deprecated', 'request_id': 'feed45ef-b405-4d5c-a9ec-abc94aa358f0', 'sha256': 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855', 'created': '2025-02-03T18:11:51.184Z', 'duration': 0.0, 'channels': 0}

Yes we are handling the results correctly. There is no error which is coming up when the connection is closed. Things are breaking when we are sending audio data from the browser. From the browser we are capturing data via the mic. As in the code shared above I have specified the audio format (webm) and encoding type (opus). This is sent to our backend written in python from where we send the data to deepgram.

Also one correction from my end, I just did another session and I received the following 3 packets from deepgram -

{'type': 'SpeechStarted', 'channel': [0, 1], 'timestamp': 0.05}
{'type': 'Results', 'channel_index': [0, 1], 'duration': 0.2075, 'start': 0.0, 'is_final': True, 'speech_final': True, 'channel': {'alternatives': [{'transcript': '', 'confidence': 0.0, 'words': []}]}, 'metadata': {'request_id': '6ec4eb27-cd23-404d-8d8e-0f4b4d6222c6', 'model_info': {'name': '2-general-nova', 'version': '2024-01-11.36317', 'arch': 'nova-2'}, 'model_uuid': '1dbdfb4d-85b2-4659-9831-16b3c76229aa'}, 'from_finalize': False}
{'type': 'Metadata', 'transaction_key': 'deprecated', 'request_id': '6ec4eb27-cd23-404d-8d8e-0f4b4d6222c6', 'sha256': '07cfe2aeb59638704780dd505ad78bb393f93a3e03af1bb50e71a4e529575fdd', 'created': '2025-02-03T18:50:24.427Z', 'duration': 0.2075, 'channels': 1, 'models': ['1dbdfb4d-85b2-4659-9831-16b3c76229aa'], 'model_info': {'1dbdfb4d-85b2-4659-9831-16b3c76229aa': {'name': '2-general-nova', 'version': '2024-01-11.36317', 'arch': 'nova-2'}}}

Based on the above mentioned code, do you find anything which is being done wrong? My ultimate goal is to collect audio from the browser via a microphone, send that audio to a backend written in python and then from there send the audio to deepgram for transcribing. If there is something that I am missing at the browser level implementation or if I am missing something at the python backend implementation then please do let me know!

[jkroll-deepgram]

Hi @deva-gopalani, I do see evidence of two similar but distinct issues on those two request IDs.

For the first request, feed45ef-b405-4d5c-a9ec-abc94aa358f0, Deepgram logged that invalid audio was sent - possibly empty, non-audio, or base64 encoded. MIME type was inode/x-empty. Please make sure you are sending raw, unencoded binary data.

For the second request, 6ec4eb27-cd23-404d-8d8e-0f4b4d6222c6, the audio duration is only 0.2 seconds. It appears that you sent a small amount of audio and then sent a CloseStream message, initiating Deepgram to close the websocket.

So overall, the issue lies in the audio encoding and duration that you are sending to Deepgram. If Deepgram receives little to no audio, then obviously no transcription can occur.

[olivernaaris]

I have a same issue. Also streaming from browser to backend and DeepGram in WebM. I can see that DeepGram is receiving the audio chunks, but never responds back with the transcript.

[deva-gopalani]

@jkroll-deepgram any update? Seems to be a genuine issue as @olivernaaris is also facing the same.

[olivernaaris]

I also created a issue in DeepGram discord.
My frontend uses MediaRecorder in the browser to get the audio and stream it to the backend via Socket.io. The backend is running Python and it gets the audio chunks and saves the final audio as webm file to the filesystem (for troubleshooting) and also tries to send the audio chunks to DeepGram.
The webm audio file that gets saved to the filesystem works well when i use ffprobe to see the audio file details or VLC to play the audio file. I can see from the Python backend logs that the audio chunks are being streamed to DeepGram, but no transcript is ever received back.
I created a testing script to do STT streaming with the local webm file to DeepGram and it seems to work fine, I get transcript back.
When I check DeepGram GUI logs, I can see that it's receiving the audio....but it's not clear what is the issue.
It's honestly disappointing that DeepGram API does not give you more details why the audio chunk is not accepted.

And yes, I turned on verbose logging in DeepGram SDK, added log as well to this message.

Testing script that works streaming local webm file to DeepGram:

#!/usr/bin/env python3
"""
This script demonstrates how to use the Deepgram API for live audio transcription
using a WebSocket connection. It reads an audio file in chunks and sends it to
Deepgram for real-time transcription, printing the results to the console.

Usage:
    1. Set the DEEPGRAM_API_KEY environment variable or configure it in your .env file.
    2. Place an audio file in the development_audio directory and update the AUDIO_FILE_PATH variable.
    3. Run the script: `python deepgram_test_class.py`
"""

import asyncio
import logging
from deepgram import DeepgramClient, LiveOptions, LiveTranscriptionEvents, DeepgramClientOptions
from deepgram.utils import verboselogs

# Configure logging
logging.basicConfig(level=logging.DEBUG)
logger = logging.getLogger(__name__)

# Replace with your Deepgram API key and audio file path.
API_KEY = ""
AUDIO_FILE_PATH = "<path to your webm file>.webm"

# Define transcription options.
OPTIONS = LiveOptions(
    model="nova-2-general",
    language="en",
    smart_format=True,
    interim_results=True,
    punctuate=True,
    profanity_filter=True
)


class DeepgramTranscriber:
    def __init__(self, api_key: str, options: LiveOptions):
        """
        Initialize the transcriber with a Deepgram API key and transcription options.
        The Deepgram client is stored as an instance variable.
        """
        self.api_key = api_key
        self.options = options
        self.deepgram: DeepgramClient = DeepgramClient(
            api_key=self.api_key,
            config=DeepgramClientOptions(
                verbose=verboselogs.DEBUG,
                options={"keepalive": "true"}
            )
        )
        # We'll create the connection later.
        self.dg_connection = None

    # --- Event Handler Methods ---
    async def on_open(self, *args, **kwargs):
        logger.info(f"on_open called with positional args: {args}")
        logger.info(f"on_open called with keyword args: {kwargs}")
        if args:
            event_data = args[0]
            logger.info(f"Deepgram connection opened: {event_data}")

    async def on_transcript(self, *args, **kwargs):
        logger.info(f"on_transcript called with positional args: {args}")
        logger.info(f"on_transcript called with keyword args: {kwargs}")
        if args:
            result = args[0]
            try:
                transcript = result.channel.alternatives[0].transcript
                confidence = result.channel.alternatives[0].confidence
                is_final = result.is_final
                if transcript:
                    logger.info(f"Transcript: '{transcript}' (confidence: {confidence}, final: {is_final})")
            except Exception as e:
                logger.error(f"Error processing transcript: {e}")

    async def on_error(self, *args, **kwargs):
        logger.error(f"on_error called with positional args: {args}")
        logger.error(f"on_error called with keyword args: {kwargs}")
        if args:
            error = args[0]
            logger.error(f"Deepgram error: {error}")

    async def on_close(self, *args, **kwargs):
        logger.info(f"on_close called with positional args: {args}")
        logger.info(f"on_close called with keyword args: {kwargs}")
        if args:
            event_data = args[0]
            logger.info(f"Deepgram connection closed: {event_data}")

    # --- Main Method for Running the Transcription ---
    async def run(self, audio_file_path: str):
        """
        Opens the Deepgram connection, sends the audio file in chunks,
        and prints transcript events to the console.
        """
        # Create the asynchronous Deepgram WebSocket connection.
        self.dg_connection = self.deepgram.listen.asyncwebsocket.v("1")

        # Attach the event handlers.
        self.dg_connection.on(LiveTranscriptionEvents.Open, self.on_open)
        self.dg_connection.on(LiveTranscriptionEvents.Transcript, self.on_transcript)
        self.dg_connection.on(LiveTranscriptionEvents.Error, self.on_error)
        self.dg_connection.on(LiveTranscriptionEvents.Close, self.on_close)

        try:
            # Start the Deepgram connection with the specified options.
            await self.dg_connection.start(self.options)
            logger.info("Deepgram WebSocket connection started.")

            # Open the audio file and send it in chunks.
            with open(audio_file_path, "rb") as audio_file:
                while True:
                    chunk = audio_file.read(4096)  # Read in 4KB chunks.
                    if not chunk:
                        break
                    await self.dg_connection.send(chunk)
                    await asyncio.sleep(0.1)  # Slight delay to simulate real-time streaming.

            # Optionally, wait a bit to ensure all data is processed.
            await asyncio.sleep(1)
        except Exception as e:
            logger.error(f"An exception occurred: {e}")
        finally:
            # Close the connection.
            await self.dg_connection.finish()
            logger.info("Deepgram WebSocket connection finished.")


if __name__ == "__main__":
    async def main():
        transcriber = DeepgramTranscriber(API_KEY, OPTIONS)
        await transcriber.run(AUDIO_FILE_PATH)

    asyncio.run(main())

My frontend and backend app code using DeepGram, socket.io etc:

import { useRef, useState, useEffect } from "react";
import { MediaRecorder as ExtendedMediaRecorder } from "extendable-media-recorder";
import { useSttSocketContext } from "@/contexts/SttSocketContext";

export const useWebSocketRecorder = ({
  isRecording,
  setIsRecording,
  setErrorAlert,
  t,
  analyzeVolume,
  stopAllIntervals,
  hasStoppedRecordingRef,
  handleSubmit,
  ai_character_language,
  setVoiceRecordingAlert,
  getDisplayLanguage,
}: {
  isRecording: boolean;
  setIsRecording: (isRecording: boolean) => void;
  setErrorAlert: (message: string) => void;
  t: (key: string, options?: Record<string, any>) => string;
  analyzeVolume: (stream: MediaStream, audioContext: AudioContext) => void;
  stopAllIntervals: () => void;
  hasStoppedRecordingRef: React.MutableRefObject<boolean>;
  handleSubmit: (text: string) => void;
  setVoiceRecordingAlert: (message: string) => void;
  ai_character_language: string;
  getDisplayLanguage: (language: string) => string;
}) => {
  const recognitionRef = useRef<InstanceType<typeof ExtendedMediaRecorder> | null>(null);
  const silenceDurationRef = useRef(0);
  const [recordTime, setRecordTime] = useState(0);
  const [transcript] = useState("");
  const timerInterval = useRef<NodeJS.Timeout | null>(null);
  const allTextRef = useRef("");
  const confidenceScoreRef = useRef(0);
  const isConnectingRef = useRef(false);
  const finalTranscriptionReceivedRef = useRef(false);

  const { sttSocket, isSttConnected } = useSttSocketContext();

  // Helper to reset recording-related state
  const clearRecordingState = () => {
    console.log("Clearing recording state");
    allTextRef.current = "";
    confidenceScoreRef.current = 0;
    isConnectingRef.current = false;
  };

  // Returns the first supported MIME type for audio recording
  const getPreferredMimeType = (): string | null => {
    const mimeTypes = [
      "audio/webm;codecs=opus",
      "audio/mp4;codecs=mp4a.40.2",
      "audio/webm",
      "audio/ogg;codecs=opus",
    ];
    return mimeTypes.find((mime) => MediaRecorder.isTypeSupported(mime)) || null;
  };

  const startRecording = async () => {
    if (isRecording || isConnectingRef.current) return;
    if (!isSttConnected || !sttSocket) {
      console.error("STT Socket is not connected");
      setErrorAlert(t("websocketNotReady"));
      return;
    }

    isConnectingRef.current = true;
    allTextRef.current = "";
    confidenceScoreRef.current = 0;
    finalTranscriptionReceivedRef.current = false;

    try {
      localStorage.setItem("isRecording", "true");

      const stream = await navigator.mediaDevices.getUserMedia({
        audio: {
          channelCount: 1,
          sampleRate: 48000,
          sampleSize: 16,
          noiseSuppression: true,
          echoCancellation: false,
          autoGainControl: false,
        },
      });
      const audioContext = new (window.AudioContext ||
        (window as any).webkitAudioContext)();

      const mimeType = getPreferredMimeType();
      if (!mimeType) {
        setErrorAlert(t("unsupportedAudioFormat"));
        setIsRecording(false);
        return;
      }

      const options = { mimeType };
      const mediaRecorder = new ExtendedMediaRecorder(stream, options);
      recognitionRef.current = mediaRecorder;

      console.log("Using MIME type:", mimeType);
      sttSocket.emit("start_stt", { mimeType });

      setIsRecording(true);
      setRecordTime(0);
      silenceDurationRef.current = 0;
      localStorage.setItem("isRecording", "true");

      // Wait for the volume visualizer to be available before starting the recorder.
      const checkCanvasInterval = setInterval(() => {
        const canvas = document.getElementById("volumeVisualizer");
        if (canvas) {
          clearInterval(checkCanvasInterval);
          mediaRecorder.start(250);
          analyzeVolume(stream, audioContext);
        }
      }, 200);

      // Use async/await with arrayBuffer() for data conversion
      mediaRecorder.ondataavailable = async (event) => {
        if (event.data.size > 0 && sttSocket && isSttConnected) {
          const buffer = await event.data.arrayBuffer();
          sttSocket.emit("listen", {
            chunk: buffer,
            mimeType,
          });
        }
      };

      hasStoppedRecordingRef.current = false;

      if (timerInterval.current) clearInterval(timerInterval.current);
      timerInterval.current = setInterval(() => setRecordTime((prevTime) => prevTime + 10), 10);

      mediaRecorder.onstop = () => {
        setRecordTime(0);
        // Any additional cleanup for volume checking can be done here.
      };
    } catch (error: any) {
      const errMsg = error instanceof Error ? error.message : String(error);
      const errorMessage =
        errMsg === "Permission denied"
          ? t("permissionDenied")
          : `${t("errorAccessingMicrophone")} ${errMsg}`;
      setErrorAlert(errorMessage);
      setIsRecording(false);
    }
  };

  const handleSaveRecording = async () => {
    recognitionRef.current?.stop();
    hasStoppedRecordingRef.current = true;
    localStorage.setItem("isRecording", "false");

    // Wait for the final transcription signal or timeout after 3 seconds.
    await new Promise<void>((resolve) => {
      const interval = setInterval(() => {
        if (finalTranscriptionReceivedRef.current) {
          console.log("Final transcription received early, proceeding.");
          clearInterval(interval);
          resolve();
        }
      }, 100);
      setTimeout(() => {
        console.log("Timeout reached, proceeding anyway.");
        clearInterval(interval);
        resolve();
      }, 3000);
    });

    sttSocket?.emit("stop_stt");
    setIsRecording(false);
    stopAllIntervals();

    // Process the transcription result.
    if (allTextRef.current && confidenceScoreRef.current > 0.01) {
      console.log("Submitting transcription:", allTextRef.current);
      handleSubmit(allTextRef.current);
    } else {
      setVoiceRecordingAlert(
        t("lowConfidenceScore", {
          language: getDisplayLanguage(ai_character_language),
        })
      );
    }
    clearRecordingState();
  };

  const handleCancelRecording = () => {
    recognitionRef.current?.stop();
    hasStoppedRecordingRef.current = true;
    localStorage.setItem("isRecording", "false");

    sttSocket?.emit("stop_stt");

    setIsRecording(false);
    stopAllIntervals();
    clearRecordingState();
  };

  // Listen for STT socket reconnection attempts.
  useEffect(() => {
    if (!sttSocket) return;
    const handleReconnect = (attempt: number) => {
      console.log(`Reconnection attempt ${attempt}`);
      setErrorAlert(t("reconnectingAttempt", { attempt }));
    };

    sttSocket.on("reconnect_attempt", handleReconnect);
    return () => {
      sttSocket.off("reconnect_attempt", handleReconnect);
    };
  }, [sttSocket, t, setErrorAlert]);

  return {
    isRecording,
    recordTime,
    transcript,
    startRecording,
    handleCancelRecording,
    handleSaveRecording,
  };
};

import asyncio
import logging
import socketio
from typing import Dict, Optional, Any
from dataclasses import dataclass, field
from deepgram import DeepgramClient, LiveOptions, LiveTranscriptionEvents, DeepgramClientOptions
from deepgram.utils import verboselogs
import time

# Custom imports
from app.dependencies.subscription_dependency import require_ws_active_subscription
from app.socket.socketio_utils import MockWebsocket
from app.core.config import get_settings

settings = get_settings()
logger = logging.getLogger(__name__)

AUDIO_OUTPUT_DIR = "development_audio"

@dataclass
class STTSession:
    """
    Represents a single Speech-to-Text session, holding session-specific data.

    Attributes:
        sid (str): The session ID of the client.
        is_user_transcription_enabled (bool): Flag to indicate if user-initiated transcription is enabled and actively processing audio.
        state (dict): A dictionary to store transcription state, including full transcription, confidence scores, etc.
        connection_state (str): The current state of the Deepgram WebSocket connection (e.g., "connecting", "connected", "closed").
        dg_connection (Optional[Any]): The Deepgram WebSocket connection for the session.
        deepgram_ws_connection_ready (asyncio.Event): An event to signal when the connection is ready.
        mime_type (Optional[str]): The MIME type of the audio data.
        file_extension (str): The file extension for saving audio data.
        recorded_audio_data: Optional[bytes] = None  # Single source of truth for audio data
    """
    sid: str = field(default_factory=str)
    # Flag to indicate if user-initiated transcription is enabled and actively processing audio.
    is_user_transcription_enabled: bool = True
    state: dict = field(default_factory=lambda: {
        "full_transcription": "",
        "confidence_sum": 0.0,
        "confidence_count": 0,
        "current_confidence": 0.0
    })
    connection_state: str = "disconnected"  # [connecting, connected, closing, closed]
    dg_connection: Optional[Any] = None # DeepGram object
    deepgram_ws_connection_ready: asyncio.Event = field(default_factory=asyncio.Event) # DeepGram is ready for traffic flag
    recorded_audio_data: Optional[bytearray] = None  # Single source of truth for audio data
    mime_type: Optional[str] = None # Mime type coming from the client
    file_extension: str = 'webm'  # Local file extension for debugging

class STTNamespace(socketio.AsyncNamespace):
    """
    A Socket.IO namespace for handling Speech-to-Text (STT) functionality.

    This class manages WebSocket connections with clients, establishes connections with Deepgram,
    and handles audio streaming and transcription. It uses asyncio for asynchronous operations.
    """
    def __init__(self, namespace=None):
        """
        Initializes the STTNamespace with session management and Deepgram client.

        Args:
            namespace (str, optional): The namespace for this handler. Defaults to None.
        """
        super().__init__(namespace)
        # Dictionary to store active STT sessions, mapping session IDs (sids) to STTSession objects.
        self.sessions: Dict[str, STTSession] = {}
        # Deepgram client for interacting with the Deepgram API.
        self.deepgram: DeepgramClient = DeepgramClient(
            api_key=settings.DEEPGRAM_KEY.get_secret_value(),
            config=DeepgramClientOptions(
                verbose=verboselogs.SPAM,
                options={"keepalive": "true"}
                ),
        )

    async def on_connect(self, sid: str, environ: dict, auth: dict) -> None:
        """
        Handles a new client connection to the STT namespace.

        Authenticates the client using subscription dependency.

        Args:
            sid (str): The session ID of the client.
            environ (dict): The environment variables of the connection.
            auth (dict): The authentication data of the connection.
        """
        logger.info(f"STTNamespace on_connect: sid={sid}")
        try:
            mock_ws = MockWebsocket(sid, environ, auth)
            await require_ws_active_subscription(mock_ws)
            logger.info(f"Client {sid} authenticated successfully for STT.")
        except Exception as e:
            logger.error(f"Auth failed: {str(e)}")

    async def on_disconnect(self, sid: str) -> None:
        """Handle client disconnection with proper resource cleanup"""
        logger.info(f"Client disconnected from STT namespace: {sid}")
        
        try:
            if session := self.sessions.pop(sid, None):
                logger.info(f"Starting cleanup for session {sid}")
                
                # Cleanup priorities
                cleanup_tasks = [
                    self._close_deepgram_connection(session),
                    self._clear_buffers(session)
                ]
                
                await asyncio.gather(*cleanup_tasks)
                logger.info(f"Session {sid} resources released")
                
        except Exception as e:
            logger.error(f"Disconnect error for {sid}: {str(e)}")
        finally:
            logger.info(f"Disconnect process completed for {sid}")

    @staticmethod
    async def _close_deepgram_connection(session: STTSession) -> None:
        """Safely close Deepgram connection with timeout"""
        if session.dg_connection and session.is_user_transcription_enabled:
            try:
                logger.info("Closing Deepgram connection")
                await asyncio.wait_for(
                    session.dg_connection.finish(),
                    timeout=5
                )
            except (asyncio.TimeoutError, Exception) as e:
                logger.warning(f"Error closing Deepgram connection: {str(e)}")
            finally:
                session.dg_connection = None
                session.deepgram_ws_connection_ready.clear()

    @staticmethod
    def _clear_buffers(session: STTSession) -> None:
        """Clear memory buffers"""
        try:
            logger.info("Clearing audio buffers")
            session.recorded_audio_data = None
        except Exception as e:
            logger.error(f"Error clearing buffers: {str(e)}")

    async def on_start_stt(self, sid: str, data: Optional[dict] = None) -> None:
        """
        Initializes a new STT session for a client.

        Creates a new STTSession object, establishes a Deepgram WebSocket connection,
        and sets up event handlers for the connection.

        Args:
            sid (str): The session ID of the client starting the STT session.
            data (Optional[dict]): Additional data from the client.
        """
        try:
            logger.info(f"Starting STT session for {sid}")
            
            # Clean up any existing session
            if existing_session := self.sessions.get(sid):
                logger.info(f"Cleaning up existing session for {sid}")
                if existing_session.dg_connection:
                    await existing_session.dg_connection.finish()
                del self.sessions[sid]

            # Get and parse MIME type
            mime_type = data.get('mimeType') if data else 'audio/webm;codecs=opus'
            
            try:
                codec, container = await self.parse_mime_type(mime_type)
            except ValueError as e:
                logger.error(f"Invalid MIME type: {str(e)}")
                await self.emit("error", {"message": str(e)}, to=sid)
                return
            
            # Configure Deepgram options
            options = LiveOptions(
                model="nova-2-general",
                smart_format=True,
                interim_results=True,
                language="en",
                #encoding=codec,
                #sample_rate=48000,
                #channels=1,
            )

            # Create new session with sid
            session = STTSession(sid=sid)
            session.mime_type = mime_type
            session.file_extension = container
            self.sessions[sid] = session
            logger.info(f"Session created and stored for sid: {sid}")

            # Create the Deepgram connection
            session.dg_connection = self.deepgram.listen.asyncwebsocket.v("1")

            # Register event handlers directly on the session connection
            session.dg_connection.on(LiveTranscriptionEvents.Open, 
                                    lambda *a, **kw: self.on_open(sid, *a, **kw))
            session.dg_connection.on(LiveTranscriptionEvents.Close,
                                    lambda *a, **kw: self.on_close(sid=sid, *a, **kw))
            session.dg_connection.on(LiveTranscriptionEvents.Error,
                                    lambda *a, **kw: self.on_error(sid=sid, *a, **kw))
            session.dg_connection.on(LiveTranscriptionEvents.Transcript,
                                    lambda *a, **kw: self.on_message(sid=sid, *a, **kw))
            session.dg_connection.on(LiveTranscriptionEvents.UtteranceEnd,
                                    lambda *a, **kw: self.on_utterance_end(sid=sid, *a, **kw))
            session.dg_connection.on(LiveTranscriptionEvents.Metadata,
                                    lambda *a, **kw: self.on_metadata(sid=sid, *a, **kw))
            session.dg_connection.on(LiveTranscriptionEvents.SpeechStarted,
                                    lambda *a, **kw: self.on_speech_started(sid=sid, *a, **kw))
            session.dg_connection.on(LiveTranscriptionEvents.Unhandled,
                                    lambda *a, **kw: self.on_unhandled(sid=sid, *a, **kw))

            # Set the connection state to "connecting".
            session.connection_state = "connecting"
            try:
                # Start the Deepgram WebSocket connection.
                await session.dg_connection.start(options)
                # Set the connection state to "connected" if successful.
                session.connection_state = "connected"
                session.deepgram_ws_connection_ready.set()
                logger.info(f"Deepgram connection established for {sid}")
            except Exception as e:
                # Set the connection state to "closed" if there's an error.
                session.connection_state = "closed"
                await self.emit("error", {"message": "Connection failed"}, to=sid)
                return

            # Wait for Deepgram connection to open
            await asyncio.wait_for(session.deepgram_ws_connection_ready.wait(), timeout=5)
            logger.info(f"Deepgram fully connected for {sid}")
            # Emit a "session_ready" event to the client.
            await self.emit("session_ready", to=sid)
            # Introduce a small delay after connection is ready, before allowing audio to be sent
            await asyncio.sleep(0.1) # 100ms delay

        except Exception as e:
            logger.error(f"Session start failed for {sid}: {e}")
            await self.emit("error", {"message": "Session initialization failed"}, to=sid)

    async def on_listen(self, sid: str, data: dict) -> None:
        """Validate incoming audio chunks"""
        if not data or 'chunk' not in data or 'mimeType' not in data:
            logger.error("Invalid audio chunk format")
            await self.emit("error", {"message": "Invalid audio format"}, to=sid)
            return

        session = self.sessions.get(sid)
        if not session:
            logger.error(f"No session found for sid: {sid} in on_listen. Current sessions: {list(self.sessions.keys())}")
            return # Add this return to prevent further errors

        if session and data['mimeType'] != session.mime_type:
            logger.error("MIME type mismatch during session")
            await self.emit("error", {"message": "Audio format changed mid-session"}, to=sid)
            return

        # Continue with existing processing logic
        # Validate the incoming data.
        if not data or 'chunk' not in data:
            logger.warning(f"Invalid chunk structure from {sid}")
            return
        
        chunk = data['chunk']
        if not chunk or not isinstance(chunk, bytes):
            return

        try:
            await self.send_audio_chunk(sid, data)
        except Exception as e:
            logger.error(f"Chunk error for {sid}: {e}")
            await self.emit("error", {"message": str(e)}, to=sid)

    async def on_stop_stt(self, sid: str) -> None:
        """
        Stops an active STT session for a client.

        Closes the Deepgram WebSocket connection and cleans up session resources.

        Args:
            sid (str): The session ID of the client stopping the STT session.
        """
        logger.info(f"Stopping STT session for {sid}")
        if not (session := self.sessions.get(sid)):
            return

        try:
            if session.dg_connection and session.is_user_transcription_enabled:
                logger.info(f"Finishing STT session with DeepGram for {sid}")
                session.is_user_transcription_enabled = False
                await session.dg_connection.finish()
                session.deepgram_ws_connection_ready.clear()
                
                if settings.ENVIRONMENT == "development" and session.recorded_audio_data:
                    filename = await write_audio_blob_to_file(
                        session.recorded_audio_data, 
                        session,
                        sid
                    )
                    logger.info(f"Debug audio saved to: {filename}")

                session.recorded_audio_data = None

        except Exception as e:
            logger.error(f"Error stopping STT session for {sid}: {e}")

    # Define event handler closures here
    async def on_open(self, *args, **kwargs):
        """Handles the Deepgram WebSocket 'open' event."""
        logger.info(f"on_open called! Args: {args}, Kwargs: {kwargs}")
        sid = kwargs.get("sid")  # extra parameter passed via partial
        session = self.sessions.get(sid)
        # Signal that the Deepgram WebSocket connection is ready.
        session.deepgram_ws_connection_ready.set()
        await self.emit("open", {"event": "open", "message": str(open)}, to=sid)

    async def on_metadata(self, *args, **kwargs):
        """Handles the Deepgram WebSocket 'metadata' event."""
        logger.info(f"on_metadata called! Args: {args}, Kwargs: {kwargs}")

    async def on_speech_started(self, *args, **kwargs):
        """Handles the Deepgram WebSocket 'speech_started' event."""
        logger.info(f"on_speech_started called! Args: {args}, Kwargs: {kwargs}")

    async def on_close(self, *args, **kwargs):
        """Handles the Deepgram WebSocket 'close' event."""
        logger.info(f"on_close called! Args: {args}, Kwargs: {kwargs}")

    async def on_error(self, *args, **kwargs):
        """Handles the Deepgram WebSocket 'error' event."""
        logger.info(f"on_error called! Args: {args}, Kwargs: {kwargs}")

    async def on_unhandled(self, *args, **kwargs):
        """Handles the Deepgram WebSocket 'unhandled' event."""
        logger.info(f"on_unhandled called! Args: {args}, Kwargs: {kwargs}")

    async def on_message(self, *args, **kwargs):
        """Handles Deepgram transcript results with proper argument unpacking."""
        logger.info(f"Transcript received - args: {args}, kwargs: {kwargs}")
        
        try:
            # Extract result from keyword arguments
            result = kwargs.get("result")
            sid = kwargs.get("sid")
            if not result:
                logger.error("No result in transcript message")
                return

            if not (session := self.sessions.get(sid)):
                return

            # Extract transcript and confidence from the result.
            transcript = result.channel.alternatives[0].transcript
            confidence = result.channel.alternatives[0].confidence
            if transcript:
                logger.debug(f"Received transcript: '{transcript}' (confidence: {confidence})")
                if result.is_final:
                    logger.info(f"Final transcript received: '{transcript}'")
                    # Append the final transcript to the full transcription.
                    session.state["full_transcription"] += f" {transcript}"
                    # Update the confidence sum and count for overall confidence score.
                    session.state["confidence_sum"] += confidence
                    session.state["confidence_count"] += 1
                # Update the current transcript and confidence.
                session.state["transcript"] = transcript
                session.state["current_confidence"] = confidence
                # Calculate the overall confidence score.
                overall_confidence_score = (
                    session.state["confidence_sum"] / session.state["confidence_count"]
                    if session.state["confidence_count"] > 0
                    else confidence
                )
                # Emit the transcript to the client.
                await self.emit(
                    "transcript",
                    {
                        "event": "transcript",
                        "message": transcript,
                        "full_transcription": session.state["full_transcription"].strip(),
                        "overall_confidence_score": overall_confidence_score,
                        "is_final": result.is_final,
                        "speech_final": result.speech_final
                    },
                    to=sid
                )
        except Exception as e:
            logger.info(f"on_message failed with: {e}")

    async def on_utterance_end(self, *args, **kwargs):
        """Handles the Deepgram WebSocket 'utterance_end' event."""
        logger.info(f"on_utterance_end called! Args: {args}, Kwargs: {kwargs}")
        sid = kwargs.get("sid")

        if session := self.sessions.get(sid):
            # Add finalization flag
            await self.emit("utterance_end", {
                "final_transcript": session.state["full_transcription"],
                "is_final": True
            }, to=sid)

    async def send_audio_chunk(self, sid: str, data: dict) -> None:
        """Safely send audio chunks with connection state validation"""
        logger.info(f"send_audio_chunk: sid={sid}")
        
        if not (session := self.sessions.get(sid)):
            logger.error(f"No active session for {sid}")
            return

        try:
            chunk = data['chunk']
            
            # Development audio recording - write to buffer only
            if settings.ENVIRONMENT == "development":
                if session.recorded_audio_data is None:
                    session.recorded_audio_data = bytearray()
                session.recorded_audio_data.extend(chunk)

            # Send to Deepgram
            await session.dg_connection.send(chunk)

        except Exception as e:
            logger.error(f"Audio chunk error: {str(e)}")
            await self.emit('error', {'message': 'Audio processing error'}, to=sid)

    @staticmethod
    async def parse_mime_type(mime_type: str) -> tuple[str, str]:
        """
        Parse audio codec and container from MIME type string
        Returns: (codec, container)
        """
        # Default fallbacks
        codec = 'opus'
        container = 'webm'

        if not mime_type:
            return codec, container

        mime_type = mime_type.lower()
        
        # WebM variants
        if 'webm' in mime_type:
            container = 'webm'
            codec = 'opus' if 'opus' in mime_type else 'pcm'
        
        # MP4 variants
        elif 'mp4' in mime_type or 'mp4a.40.2' in mime_type:
            container = 'mp4'
            codec = 'aac'
        
        # Add more formats as needed
        else:
            raise ValueError(f"Unsupported MIME type: {mime_type}")

        return codec, container

class AudioWriter:
    """Base class for audio file writers"""
    def __init__(self):
        self.initialized = False
        self.file_extension = 'dat'

    async def initialize(self, sample_rate: int, channels: int):
        """Base initialization (mark as initialized)"""
        self.initialized = True  # Add this base implementation

class WebMOpusWriter(AudioWriter):
    """WebM/Opus audio writer"""
    def __init__(self):
        super().__init__()
        self.file_extension = 'webm'
        self._header = None

    async def initialize(self, sample_rate: int, channels: int):
        await super().initialize(sample_rate, channels)
        # Build proper EBML header structure
        self._header = bytes([
            0x1A, 0x45, 0xDF, 0xA3,  # EBML header ID
            0x9F,                    # Header size (31 bytes)
            0x42, 0x86, 0x81, 0x01,  # EBMLVersion (1)
            0x42, 0xF7, 0x81, 0x01,  # EBMLReadVersion (1)
            0x42, 0xF2, 0x81, 0x04,  # EBMLMaxIDLength (4)
            0x42, 0xF3, 0x81, 0x08,  # EBMLMaxSizeLength (8)
            0x42, 0x82, 0x84, 0x77, 0x65, 0x62, 0x6D,  # DocType ("webm")
            0x42, 0x87, 0x81, 0x04,  # DocTypeVersion (4)
            0x42, 0x85, 0x81, 0x02   # DocTypeReadVersion (2)
        ])

    def write_chunk(self, chunk: bytes) -> bytes:
        """Create valid WebM container structure"""
        return self._header + chunk

class MP4AACWriter(AudioWriter):
    """MP4/AAC audio writer with proper header"""
    def __init__(self):
        super().__init__()
        self.file_extension = 'mp4'
        self._init_segment = None
        self._creation_time = int(time.time())
        self._timescale = 1000  # 1ms units

    async def initialize(self, sample_rate: int, channels: int):
        await super().initialize(sample_rate, channels)
        # Build MP4 header structure
        header = bytearray()
        
        # ftyp box (file type)
        ftyp = bytearray()
        ftyp.extend(len_to_bytes(28))  # Box size
        ftyp.extend(b'ftyp')           # Box type
        ftyp.extend(b'mp42')           # Major brand
        ftyp.extend(b'\x00\x00\x00\x00')  # Minor version
        ftyp.extend(b'mp42isom')       # Compatible brands
        
        # moov box (movie metadata)
        moov = bytearray()
        
        # mvhd box (movie header)
        mvhd = bytearray()
        mvhd.extend(len_to_bytes(108))
        mvhd.extend(b'mvhd')
        mvhd.extend(b'\x00')          # Version 0
        mvhd.extend(b'\x00\x00\x00')   # Flags
        mvhd.extend(self._creation_time.to_bytes(4, 'big'))  # Creation time
        mvhd.extend(self._creation_time.to_bytes(4, 'big'))  # Modification time
        mvhd.extend(self._timescale.to_bytes(4, 'big'))      # Timescale
        mvhd.extend(b'\x00\x00\x00\x00')  # Duration (0 for placeholder)
        mvhd.extend(b'\x00\x01\x00\x00')  # Rate (1.0)
        mvhd.extend(b'\x01\x00')         # Volume (1.0)
        mvhd.extend(b'\x00\x00')         # Reserved
        mvhd.extend(b'\x00\x00\x00\x00')  # Reserved
        mvhd.extend(b'\x00\x00\x00\x00')  # Matrix
        mvhd.extend(b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00')
        mvhd.extend(b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00')
        mvhd.extend(b'\x00\x00\x00\x00')  # Preview time
        mvhd.extend(b'\x00\x00\x00\x00')  # Preview duration
        mvhd.extend(b'\x00\x00\x00\x00')  # Poster time
        mvhd.extend(b'\x00\x00\x00\x00')  # Selection time
        mvhd.extend(b'\x00\x00\x00\x00')  # Selection duration
        mvhd.extend(b'\x00\x00\x00\x00')  # Current time
        mvhd.extend(b'\x00\x00\x00\x02')  # Next track ID
        
        # trak box (track)
        trak = bytearray()
        
        # tkhd box (track header)
        tkhd = bytearray()
        tkhd.extend(len_to_bytes(92))
        tkhd.extend(b'tkhd')
        tkhd.extend(b'\x00')          # Version 0
        tkhd.extend(b'\x00\x00\x0f')  # Flags (track enabled, in preview, in movie)
        tkhd.extend(self._creation_time.to_bytes(4, 'big'))  # Creation time
        tkhd.extend(self._creation_time.to_bytes(4, 'big'))  # Modification time
        tkhd.extend(b'\x00\x00\x00\x00')  # Track ID (will be updated)
        tkhd.extend(b'\x00\x00\x00\x00')  # Reserved
        tkhd.extend(b'\x00\x00\x00\x00')  # Duration
        tkhd.extend(b'\x00\x00\x00\x00')  # Reserved
        tkhd.extend(b'\x00\x00\x00\x00')  # Layer
        tkhd.extend(b'\x00\x00')         # Alternate group
        tkhd.extend(b'\x01\x00')         # Volume (1.0)
        tkhd.extend(b'\x00\x00')         # Reserved
        tkhd.extend(b'\x00\x01\x00\x00\x00\x00\x00\x00')  # Matrix
        tkhd.extend(b'\x00\x00\x00\x00\x00\x01\x00\x00')
        tkhd.extend(b'\x00\x00\x00\x00\x00\x00\x00\x00')
        tkhd.extend(b'\x00\x01\x00\x00')  # Track width
        tkhd.extend(b'\x00\x01\x00\x00')  # Track height
        
        # Build remaining structure
        mdia = self._create_mdia_box(sample_rate)
        trak.extend(tkhd)
        trak.extend(mdia)
        moov.extend(mvhd)
        moov.extend(trak)
        
        # Combine all boxes
        header.extend(ftyp)
        header.extend(len_to_bytes(8 + len(moov)))  # moov size
        header.extend(b'moov')
        header.extend(moov)
        
        self._init_segment = bytes(header)

    def _create_mdia_box(self, sample_rate: int) -> bytearray:
        """Create media information box with audio-specific data"""
        mdia = bytearray()
        
        # mdhd box (media header)
        mdhd = bytearray()
        mdhd.extend(len_to_bytes(32))
        mdhd.extend(b'mdhd')
        mdhd.extend(b'\x00')          # Version 0
        mdhd.extend(b'\x00\x00\x00')  # Flags
        mdhd.extend(self._creation_time.to_bytes(4, 'big'))  # Creation time
        mdhd.extend(self._creation_time.to_bytes(4, 'big'))  # Modification time
        mdhd.extend(self._timescale.to_bytes(4, 'big'))      # Timescale
        mdhd.extend(b'\x00\x00\x00\x00')  # Duration
        mdhd.extend(b'\x55\xC4')         # Language (und)
        mdhd.extend(b'\x00\x00')         # Quality
        
        # hdlr box (handler)
        hdlr = bytearray()
        hdlr.extend(len_to_bytes(33))
        hdlr.extend(b'hdlr')
        hdlr.extend(b'\x00\x00\x00\x00')  # Version + flags
        hdlr.extend(b'soun')              # Handler type (sound)
        hdlr.extend(b'\x00\x00\x00\x00')  # Reserved
        hdlr.extend(b'\x00\x00\x00\x00')  # Reserved
        hdlr.extend(b'SoundHandler')      # Handler name
        hdlr.extend(b'\x00')              # Null terminator
        
        # minf box (media information)
        minf = bytearray()
        minf.extend(self._create_sound_media_header())
        minf.extend(self._create_data_info_box())
        minf.extend(self._create_sample_table_box(sample_rate))
        
        mdia.extend(mdhd)
        mdia.extend(hdlr)
        mdia.extend(minf)
        return mdia

    def _create_sound_media_header(self) -> bytearray:
        """Create sound-specific media header"""
        smhd = bytearray()
        smhd.extend(len_to_bytes(16))
        smhd.extend(b'smhd')
        smhd.extend(b'\x00')          # Version 0
        smhd.extend(b'\x00\x00\x00')  # Flags
        smhd.extend(b'\x00\x00')      # Balance
        smhd.extend(b'\x00\x00')      # Reserved
        return smhd

    def _create_data_info_box(self) -> bytearray:
        """Create data information box"""
        dinf = bytearray()
        dinf.extend(len_to_bytes(36))
        dinf.extend(b'dinf')
        dref = bytearray()
        dref.extend(len_to_bytes(28))
        dref.extend(b'dref')
        dref.extend(b'\x00\x00\x00\x00')  # Version + flags
        dref.extend(b'\x00\x00\x00\x01')  # Entry count
        dref.extend(len_to_bytes(12))
        dref.extend(b'url ')
        dref.extend(b'\x00\x00\x00\x01')  # Version + flags
        dinf.extend(dref)
        return dinf

    def _create_sample_table_box(self, sample_rate: int) -> bytearray:
        """Create sample table box with audio format information"""
        stbl = bytearray()
        
        # stsd box (sample description)
        stsd = bytearray()
        stsd.extend(len_to_bytes(72))
        stsd.extend(b'stsd')
        stsd.extend(b'\x00\x00\x00\x00')  # Version + flags
        stsd.extend(b'\x00\x00\x00\x01')  # Entry count
        
        # Audio sample entry
        audio_entry = bytearray()
        audio_entry.extend(len_to_bytes(56))
        audio_entry.extend(b'mp4a')      # Format
        audio_entry.extend(b'\x00\x00')  # Reserved
        audio_entry.extend(b'\x00\x01')  # Data reference index
        audio_entry.extend(b'\x00\x00\x00\x00\x00\x00')  # Reserved
        audio_entry.extend(b'\x00\x02')  # Channel count
        audio_entry.extend(b'\x00\x10')  # Sample size (16 bits)
        audio_entry.extend(b'\x00\x00')  # Pre-defined
        audio_entry.extend(b'\x00\x00')  # Reserved
        audio_entry.extend(sample_rate.to_bytes(4, 'big'))  # Sample rate
        stsd.extend(audio_entry)
        
        stbl.extend(stsd)
        stbl.extend(len_to_bytes(8))  # stts box placeholder
        stbl.extend(b'stts')
        stbl.extend(b'\x00\x00\x00\x00')  # Version + flags
        stbl.extend(b'\x00\x00\x00\x00')  # Entry count
        
        return stbl

    def write_chunk(self, chunk: bytes) -> bytes:
        """Prepend initialization segment and wrap audio in mdat box"""
        if self._init_segment:
            header = self._init_segment
            self._init_segment = None  # Only send header once
            
            # Create mdat box (media data)
            mdat_header = bytearray()
            mdat_header.extend(len_to_bytes(8 + len(chunk)))
            mdat_header.extend(b'mdat')
            
            return header + mdat_header + chunk
        return chunk

def len_to_bytes(length: int) -> bytes:
    """Convert length to big-endian bytes"""
    return length.to_bytes(4, byteorder='big')

async def get_audio_writer(mime_type: str) -> AudioWriter:
    """Factory function for audio writers"""
    if 'webm' in mime_type:
        return WebMOpusWriter()
    elif 'mp4' in mime_type:
        return MP4AACWriter()
    raise ValueError(f"Unsupported MIME type: {mime_type}")

# Modified write functions
async def write_audio_blob_to_file(audio_data: bytes, session: STTSession, sid: str) -> str:
    """Write complete audio blob to properly formatted file"""
    await ensure_audio_directory()
    writer = await get_audio_writer(session.mime_type)
    
    # Initialize with default values
    await writer.initialize(sample_rate=48000, channels=1)
    
    filename = f"{AUDIO_OUTPUT_DIR}/stt_output_{sid}.{writer.file_extension}"
    with open(filename, "wb") as f:
        processed_data = writer.write_chunk(audio_data)
        f.write(processed_data)
    logger.info(f"Audio file written: {filename}")
    return filename

async def ensure_audio_directory() -> None:
    """Create audio directory if missing"""
    import os
    if not os.path.exists(AUDIO_OUTPUT_DIR):
        os.makedirs(AUDIO_OUTPUT_DIR)

DeepGram SDK verbose logging:

2025-02-05 20:20:18 | INFO     | uvicorn.access:send:474 | ::1:63157 - "GET /api/check-user-preference-exists HTTP/1.1" 200
2025-02-05 20:20:20 | INFO     | app.socket.socketio_stt_namespace:on_start_stt:156 | Starting STT session for v8DecobeASsRlsx_AABL
2025-02-05 20:20:20 | INFO     | app.socket.socketio_stt_namespace:on_start_stt:191 | Session created and stored for sid: v8DecobeASsRlsx_AABL
Version.v ENTER
version: 1
2025-02-05 20:20:20 | INFO     | deepgram.clients.listen_router:v:165 | version: 1
path: deepgram.clients.listen.v1.websocket.async_client
2025-02-05 20:20:20 | INFO     | deepgram.clients.listen_router:v:206 | path: deepgram.clients.listen.v1.websocket.async_client
class_name: AsyncListenWebSocketClient
2025-02-05 20:20:20 | INFO     | deepgram.clients.listen_router:v:207 | class_name: AsyncListenWebSocketClient
Version.v succeeded
2025-02-05 20:20:20 | NOTICE   | deepgram.clients.listen_router:notice:153 | Version.v succeeded
Version.v LEAVE
event subscribed: Open
2025-02-05 20:20:20 | INFO     | deepgram.clients.common.v1.abstract_async_websocket:on:191 | event subscribed: Open
event subscribed: Close
2025-02-05 20:20:20 | INFO     | deepgram.clients.common.v1.abstract_async_websocket:on:191 | event subscribed: Close
event subscribed: Error
2025-02-05 20:20:20 | INFO     | deepgram.clients.common.v1.abstract_async_websocket:on:191 | event subscribed: Error
event subscribed: Results
2025-02-05 20:20:20 | INFO     | deepgram.clients.common.v1.abstract_async_websocket:on:191 | event subscribed: Results
event subscribed: UtteranceEnd
2025-02-05 20:20:20 | INFO     | deepgram.clients.common.v1.abstract_async_websocket:on:191 | event subscribed: UtteranceEnd
event subscribed: Metadata
2025-02-05 20:20:20 | INFO     | deepgram.clients.common.v1.abstract_async_websocket:on:191 | event subscribed: Metadata
event subscribed: SpeechStarted
2025-02-05 20:20:20 | INFO     | deepgram.clients.common.v1.abstract_async_websocket:on:191 | event subscribed: SpeechStarted
event subscribed: Unhandled
2025-02-05 20:20:20 | INFO     | deepgram.clients.common.v1.abstract_async_websocket:on:191 | event subscribed: Unhandled
AsyncListenWebSocketClient.start ENTER
options: {
    "interim_results": true,
    "language": "en",
    "model": "nova-2-general",
    "smart_format": true
}
2025-02-05 20:20:20 | INFO     | deepgram.clients.common.v1.abstract_async_websocket:start:101 | options: {
    "interim_results": true,
    "language": "en",
    "model": "nova-2-general",
    "smart_format": true
}
addons: None
2025-02-05 20:20:20 | INFO     | deepgram.clients.common.v1.abstract_async_websocket:start:102 | addons: None
headers: None
2025-02-05 20:20:20 | INFO     | deepgram.clients.common.v1.abstract_async_websocket:start:103 | headers: None
members: None
2025-02-05 20:20:20 | INFO     | deepgram.clients.common.v1.abstract_async_websocket:start:104 | members: None
kwargs: {}
2025-02-05 20:20:20 | INFO     | deepgram.clients.common.v1.abstract_async_websocket:start:105 | kwargs: {}
ListenWebSocketOptions switching class -> dict
2025-02-05 20:20:20 | INFO     | deepgram.clients.common.v1.abstract_async_websocket:start:126 | ListenWebSocketOptions switching class -> dict
AbstractAsyncWebSocketClient.start ENTER
addons: None
2025-02-05 20:20:20 | INFO     | deepgram.clients.common.v1.abstract_async_websocket:start:96 | addons: None
headers: None
2025-02-05 20:20:20 | INFO     | deepgram.clients.common.v1.abstract_async_websocket:start:97 | headers: None
kwargs: {}
2025-02-05 20:20:20 | INFO     | deepgram.clients.common.v1.abstract_async_websocket:start:98 | kwargs: {}
combined_options: {'interim_results': True, 'language': 'en', 'model': 'nova-2-general', 'smart_format': True}
combined_headers: {'Accept': 'application/json', 'Authorization': 'Token 646d1cedec251394d063139d82f40d01d2a172af', 'User-Agent': '@deepgram/sdk/v3.8.0 python/12.7'}
2025-02-05 20:20:20 | INFO     | app.socket.socketio_stt_namespace:send_audio_chunk:403 | send_audio_chunk: sid=v8DecobeASsRlsx_AABL
AbstractAsyncWebSocketClient.send ENTER
is_connected is False
2025-02-05 20:20:20 | NOTICE   | deepgram.clients.common.v1.abstract_async_websocket:notice:153 | is_connected is False
AbstractAsyncWebSocketClient.send LEAVE
2025-02-05 20:20:21 | INFO     | app.socket.socketio_stt_namespace:send_audio_chunk:403 | send_audio_chunk: sid=v8DecobeASsRlsx_AABL
AbstractAsyncWebSocketClient.send ENTER
is_connected is False
2025-02-05 20:20:21 | NOTICE   | deepgram.clients.common.v1.abstract_async_websocket:notice:153 | is_connected is False
AbstractAsyncWebSocketClient.send LEAVE
2025-02-05 20:20:21 | INFO     | app.socket.socketio_stt_namespace:send_audio_chunk:403 | send_audio_chunk: sid=v8DecobeASsRlsx_AABL
AbstractAsyncWebSocketClient.send ENTER
is_connected is False
2025-02-05 20:20:21 | NOTICE   | deepgram.clients.common.v1.abstract_async_websocket:notice:153 | is_connected is False
AbstractAsyncWebSocketClient.send LEAVE
2025-02-05 20:20:21 | INFO     | app.socket.socketio_stt_namespace:send_audio_chunk:403 | send_audio_chunk: sid=v8DecobeASsRlsx_AABL
AbstractAsyncWebSocketClient.send ENTER
is_connected is False
2025-02-05 20:20:21 | NOTICE   | deepgram.clients.common.v1.abstract_async_websocket:notice:153 | is_connected is False
AbstractAsyncWebSocketClient.send LEAVE
2025-02-05 20:20:21 | INFO     | app.socket.socketio_stt_namespace:send_audio_chunk:403 | send_audio_chunk: sid=v8DecobeASsRlsx_AABL
AbstractAsyncWebSocketClient.send ENTER
is_connected is False
2025-02-05 20:20:21 | NOTICE   | deepgram.clients.common.v1.abstract_async_websocket:notice:153 | is_connected is False
AbstractAsyncWebSocketClient.send LEAVE
after running thread: MainThread
after running thread: asyncio_0
number of active threads: 2
create _listening thread
2025-02-05 20:20:22 | NOTICE   | deepgram.clients.common.v1.abstract_async_websocket:notice:153 | create _listening thread
after running thread: MainThread
after running thread: asyncio_0
number of active threads: 2
AsyncListenWebSocketClient._emit ENTER
callback handlers for: Open
after running thread: MainThread
after running thread: asyncio_0
number of active threads: 2
waiting for tasks to finish...
AbstractAsyncWebSocketClient._listening ENTER
2025-02-05 20:20:22 | INFO     | app.socket.socketio_stt_namespace:on_open:310 | on_open called! Args: ('v8DecobeASsRlsx_AABL', <deepgram.clients.listen.v1.websocket.async_client.AsyncListenWebSocketClient object at 0x119d49850>, OpenResponse(type=<WebSocketEvents.Open: 'Open'>)), Kwargs: {}
after running thread: MainThread
after running thread: asyncio_0
number of active threads: 2
AsyncListenWebSocketClient._emit LEAVE
start succeeded
2025-02-05 20:20:22 | NOTICE   | deepgram.clients.common.v1.abstract_async_websocket:notice:153 | start succeeded
AbstractAsyncWebSocketClient.start LEAVE
after running thread: MainThread
after running thread: asyncio_0
number of active threads: 2
keepalive is enabled
2025-02-05 20:20:22 | NOTICE   | deepgram.clients.common.v1.abstract_async_websocket:notice:153 | keepalive is enabled
autoflush is disabled
2025-02-05 20:20:22 | NOTICE   | deepgram.clients.common.v1.abstract_async_websocket:notice:153 | autoflush is disabled
after running thread: MainThread
after running thread: asyncio_0
number of active threads: 2
start succeeded
2025-02-05 20:20:22 | NOTICE   | deepgram.clients.common.v1.abstract_async_websocket:notice:153 | start succeeded
AsyncListenWebSocketClient.start LEAVE
2025-02-05 20:20:22 | INFO     | app.socket.socketio_stt_namespace:on_start_stt:222 | Deepgram connection established for v8DecobeASsRlsx_AABL
2025-02-05 20:20:22 | INFO     | app.socket.socketio_stt_namespace:on_start_stt:231 | Deepgram fully connected for v8DecobeASsRlsx_AABL
AsyncListenWebSocketClient._keep_alive ENTER
2025-02-05 20:20:22 | INFO     | app.socket.socketio_stt_namespace:send_audio_chunk:403 | send_audio_chunk: sid=v8DecobeASsRlsx_AABL
AbstractAsyncWebSocketClient.send ENTER
send() succeeded
AbstractAsyncWebSocketClient.send LEAVE
2025-02-05 20:20:22 | INFO     | app.socket.socketio_stt_namespace:send_audio_chunk:403 | send_audio_chunk: sid=v8DecobeASsRlsx_AABL
AbstractAsyncWebSocketClient.send ENTER
send() succeeded
AbstractAsyncWebSocketClient.send LEAVE
2025-02-05 20:20:22 | INFO     | app.socket.socketio_stt_namespace:send_audio_chunk:403 | send_audio_chunk: sid=v8DecobeASsRlsx_AABL
AbstractAsyncWebSocketClient.send ENTER
send() succeeded
AbstractAsyncWebSocketClient.send LEAVE
2025-02-05 20:20:23 | INFO     | app.socket.socketio_stt_namespace:send_audio_chunk:403 | send_audio_chunk: sid=v8DecobeASsRlsx_AABL
AbstractAsyncWebSocketClient.send ENTER
send() succeeded
AbstractAsyncWebSocketClient.send LEAVE
2025-02-05 20:20:23 | INFO     | apscheduler.scheduler:remove_job:641 | Removed job 7ec707d2662648c8a563cba8e99325a8
2025-02-05 20:20:23 | INFO     | apscheduler.executors.default:run_coroutine_job:28 | Running job "send_pending_push_notifications_job (trigger: date[2025-02-05 20:20:23 WITA], next run at: 2025-02-05 20:20:23 WITA)" (scheduled at 2025-02-05 20:20:23.164670+08:00)
2025-02-05 20:20:23 | INFO     | httpx:_send_single_request:1786 | HTTP Request: GET http://127.0.0.1:54321/rest/v1/user_notifications?select=%2A&sent_at=is.null&canceled_at=is.null&or=%28send_at.is.null%2Csend_at.lte.2025-02-05T12%3A20%3A23.166644%2B00%3A00%29 "HTTP/1.1 200 OK"
2025-02-05 20:20:23 | INFO     | app.services.user_notifications:send_pending_notifications:92 | No pending notifications
2025-02-05 20:20:23 | INFO     | app.services.user_notifications:send_pending_push_notifications_job:112 | Scheduling for checking notifications to send in one minute...
2025-02-05 20:20:23 | INFO     | apscheduler.scheduler:_real_add_job:895 | Added job "send_pending_push_notifications_job" to job store "default"
2025-02-05 20:20:23 | INFO     | app.services.user_notifications:send_pending_push_notifications_job:118 | Job scheduled successfully.
2025-02-05 20:20:23 | INFO     | apscheduler.executors.default:run_coroutine_job:41 | Job "send_pending_push_notifications_job (trigger: date[2025-02-05 20:20:23 WITA], next run at: 2025-02-05 20:20:23 WITA)" executed successfully
2025-02-05 20:20:23 | INFO     | app.socket.socketio_stt_namespace:send_audio_chunk:403 | send_audio_chunk: sid=v8DecobeASsRlsx_AABL
AbstractAsyncWebSocketClient.send ENTER
send() succeeded
AbstractAsyncWebSocketClient.send LEAVE
2025-02-05 20:20:23 | INFO     | app.socket.socketio_stt_namespace:send_audio_chunk:403 | send_audio_chunk: sid=v8DecobeASsRlsx_AABL
AbstractAsyncWebSocketClient.send ENTER
send() succeeded
AbstractAsyncWebSocketClient.send LEAVE
2025-02-05 20:20:24 | INFO     | app.socket.socketio_stt_namespace:send_audio_chunk:403 | send_audio_chunk: sid=v8DecobeASsRlsx_AABL
AbstractAsyncWebSocketClient.send ENTER
send() succeeded
AbstractAsyncWebSocketClient.send LEAVE
2025-02-05 20:20:24 | INFO     | app.socket.socketio_stt_namespace:send_audio_chunk:403 | send_audio_chunk: sid=v8DecobeASsRlsx_AABL
AbstractAsyncWebSocketClient.send ENTER
send() succeeded
AbstractAsyncWebSocketClient.send LEAVE
2025-02-05 20:20:24 | INFO     | app.socket.socketio_stt_namespace:send_audio_chunk:403 | send_audio_chunk: sid=v8DecobeASsRlsx_AABL
AbstractAsyncWebSocketClient.send ENTER
send() succeeded
AbstractAsyncWebSocketClient.send LEAVE
2025-02-05 20:20:24 | INFO     | app.socket.socketio_stt_namespace:send_audio_chunk:403 | send_audio_chunk: sid=v8DecobeASsRlsx_AABL
AbstractAsyncWebSocketClient.send ENTER
send() succeeded
AbstractAsyncWebSocketClient.send LEAVE
2025-02-05 20:20:26 | INFO     | apscheduler.scheduler:remove_job:641 | Removed job 2db18e2db8ab47bf99496f418e24dd55
2025-02-05 20:20:26 | INFO     | apscheduler.executors.default:run_coroutine_job:28 | Running job "send_pending_push_notifications_job (trigger: date[2025-02-05 20:20:26 WITA], next run at: 2025-02-05 20:20:26 WITA)" (scheduled at 2025-02-05 20:20:26.455531+08:00)
2025-02-05 20:20:26 | INFO     | httpx:_send_single_request:1786 | HTTP Request: GET http://127.0.0.1:54321/rest/v1/user_notifications?select=%2A&sent_at=is.null&canceled_at=is.null&or=%28send_at.is.null%2Csend_at.lte.2025-02-05T12%3A20%3A26.458094%2B00%3A00%29 "HTTP/1.1 200 OK"
2025-02-05 20:20:26 | INFO     | app.services.user_notifications:send_pending_notifications:92 | No pending notifications
2025-02-05 20:20:26 | INFO     | app.services.user_notifications:send_pending_push_notifications_job:112 | Scheduling for checking notifications to send in one minute...
2025-02-05 20:20:26 | INFO     | apscheduler.scheduler:_real_add_job:895 | Added job "send_pending_push_notifications_job" to job store "default"
2025-02-05 20:20:26 | INFO     | app.services.user_notifications:send_pending_push_notifications_job:118 | Job scheduled successfully.
2025-02-05 20:20:26 | INFO     | apscheduler.executors.default:run_coroutine_job:41 | Job "send_pending_push_notifications_job (trigger: date[2025-02-05 20:20:26 WITA], next run at: 2025-02-05 20:20:26 WITA)" executed successfully
AsyncListenWebSocketClient.keep_alive ENTER
Sending KeepAlive...
2025-02-05 20:20:27 | NOTICE   | deepgram.clients.common.v1.abstract_async_websocket:notice:153 | Sending KeepAlive...
AbstractAsyncWebSocketClient.send ENTER
send() succeeded
AbstractAsyncWebSocketClient.send LEAVE
keep_alive succeeded
2025-02-05 20:20:27 | NOTICE   | deepgram.clients.common.v1.abstract_async_websocket:notice:153 | keep_alive succeeded
AsyncListenWebSocketClient.keep_alive LEAVE
2025-02-05 20:20:27 | INFO     | app.socket.socketio_stt_namespace:on_stop_stt:283 | Stopping STT session for v8DecobeASsRlsx_AABL
2025-02-05 20:20:27 | INFO     | app.socket.socketio_stt_namespace:on_stop_stt:289 | Finishing STT session with DeepGram for v8DecobeASsRlsx_AABL
AsyncListenWebSocketClient.finish ENTER
cancelling tasks...
AbstractAsyncWebSocketClient.finish ENTER
closing socket...
send Close...
AbstractAsyncWebSocketClient.send ENTER
send() succeeded
AbstractAsyncWebSocketClient.send LEAVE
AsyncListenWebSocketClient._emit ENTER
callback handlers for: Close
after running thread: MainThread
after running thread: asyncio_0
number of active threads: 2
waiting for tasks to finish...
2025-02-05 20:20:27 | INFO     | app.socket.socketio_stt_namespace:on_close:327 | on_close called! Args: (<deepgram.clients.listen.v1.websocket.async_client.AsyncListenWebSocketClient object at 0x119d49850>,), Kwargs: {'sid': 'v8DecobeASsRlsx_AABL', 'close': CloseResponse(type=<WebSocketEvents.Close: 'Close'>)}
after running thread: MainThread
after running thread: asyncio_0
number of active threads: 2
AsyncListenWebSocketClient._emit LEAVE
data type: <class 'str'>
Text data received
AsyncListenWebSocketClient._process_text ENTER
Text data received
response_type: Metadata, data: {'type': 'Metadata', 'transaction_key': 'deprecated', 'request_id': 'e404868e-a0f4-411d-bce9-6539b472f637', 'sha256': '7f831f67f7276a83f49f9f3e2f4cda15902056ddf4778e0d67efb8a38ccfebdf', 'created': '2025-02-05T12:20:12.238Z', 'duration': 0.0, 'channels': 0}
MetadataResponse: {
    "type": "Metadata",
    "transaction_key": "deprecated",
    "request_id": "e404868e-a0f4-411d-bce9-6539b472f637",
    "sha256": "7f831f67f7276a83f49f9f3e2f4cda15902056ddf4778e0d67efb8a38ccfebdf",
    "created": "2025-02-05T12:20:12.238Z",
    "duration": 0.0,
    "channels": 0
}
AsyncListenWebSocketClient._emit ENTER
callback handlers for: Metadata
after running thread: MainThread
after running thread: asyncio_0
number of active threads: 2
waiting for tasks to finish...
2025-02-05 20:20:28 | INFO     | app.socket.socketio_stt_namespace:on_metadata:319 | on_metadata called! Args: (<deepgram.clients.listen.v1.websocket.async_client.AsyncListenWebSocketClient object at 0x119d49850>,), Kwargs: {'sid': 'v8DecobeASsRlsx_AABL', 'metadata': MetadataResponse(type='Metadata', transaction_key='deprecated', request_id='e404868e-a0f4-411d-bce9-6539b472f637', sha256='7f831f67f7276a83f49f9f3e2f4cda15902056ddf4778e0d67efb8a38ccfebdf', created='2025-02-05T12:20:12.238Z', duration=0.0, channels=0, models=None, model_info=None, extra=None)}
after running thread: MainThread
after running thread: asyncio_0
number of active threads: 2
AsyncListenWebSocketClient._emit LEAVE
_process_text Succeeded
2025-02-05 20:20:28 | NOTICE   | deepgram.clients.common.v1.abstract_async_websocket:notice:153 | _process_text Succeeded
AsyncListenWebSocketClient._process_text LEAVE
_listening Succeeded
2025-02-05 20:20:28 | NOTICE   | deepgram.clients.common.v1.abstract_async_websocket:notice:153 | _listening Succeeded
AbstractAsyncWebSocketClient._listening LEAVE
closing socket...
send Close...
AbstractAsyncWebSocketClient.send ENTER
send() exiting gracefully: 1000
2025-02-05 20:20:28 | NOTICE   | deepgram.clients.common.v1.abstract_async_websocket:notice:153 | send() exiting gracefully: 1000
AbstractAsyncWebSocketClient.send LEAVE
AsyncListenWebSocketClient._emit ENTER
callback handlers for: Close
after running thread: MainThread
after running thread: asyncio_0
number of active threads: 2
waiting for tasks to finish...
2025-02-05 20:20:28 | INFO     | app.socket.socketio_stt_namespace:on_close:327 | on_close called! Args: (<deepgram.clients.listen.v1.websocket.async_client.AsyncListenWebSocketClient object at 0x119d49850>,), Kwargs: {'sid': 'v8DecobeASsRlsx_AABL', 'close': CloseResponse(type=<WebSocketEvents.Close: 'Close'>)}
after running thread: MainThread
after running thread: asyncio_0
number of active threads: 2
AsyncListenWebSocketClient._emit LEAVE
clean up socket...
socket.wait_closed...
cancelling tasks...
before running thread: MainThread
before running thread: asyncio_0
number of active threads: 2
processing _listen_thread cancel...
2025-02-05 20:20:28 | NOTICE   | deepgram.clients.common.v1.abstract_async_websocket:notice:153 | processing _listen_thread cancel...
tasks cancelled error:
2025-02-05 20:20:28 | ERROR    | deepgram.clients.common.v1.abstract_async_websocket:finish:473 | tasks cancelled error:
AbstractAsyncWebSocketClient.finish LEAVE
before running thread: MainThread
before running thread: asyncio_0
number of active threads: 2
processing _keep_alive_thread cancel...
2025-02-05 20:20:28 | NOTICE   | deepgram.clients.common.v1.abstract_async_websocket:notice:153 | processing _keep_alive_thread cancel...
tasks cancelled error:
2025-02-05 20:20:28 | ERROR    | deepgram.clients.common.v1.abstract_async_websocket:finish:558 | tasks cancelled error:
AsyncListenWebSocketClient.finish LEAVE
2025-02-05 20:20:28 | INFO     | app.socket.socketio_stt_namespace:write_audio_blob_to_file:720 | Audio file written: development_audio/stt_output_v8DecobeASsRlsx_AABL.webm
2025-02-05 20:20:28 | INFO     | app.socket.socketio_stt_namespace:on_stop_stt:300 | Debug audio saved to: development_audio/stt_output_v8DecobeASsRlsx_AABL.webm

[jpvajda]

@olivernaaris let's keep this all on one thread please. We'll try to assist both you and @deva-gopalani here on Github Discussions.

Please keep in mind this is Community Support and we our best to provide assistance when we can.

Pinging the Deepgram Team directly and posting multiple threads on the same topic isn't ideal and actually violates our Community Code of Conduct

So if you can follow these guidelines we'd appreciate it.

Thank you!

[olivernaaris]

Could you please provide an update then what we could do to resolve the issue?

[Tayyab-Ahmad-44]

Hey @jkroll-deepgram @olivernaaris , and @deva-gopalani
Hope you're all doing well.
I'm experiencing the same issue of not receiving any transcription. If you've identified the cause or found a potential solution, could you please share it with me?