How to use a 3rd party TTS in Voice Agent

[GabrielCNogueira]

I need Voice Agent to speak Portuguese and, as Deepgram's TTS does not currently supports it, I'm trying to use Cartesia (as listed in the https://developers.deepgram.com/docs/voice-agent-tts-models)

I replaced the speak.model by this:

"speak": {
    "provider": "cartesia",
    "voice_id": "unique_voice_id"
}

But I still get the default aura-asteria-en model in the response.

Is there some other configuration I need to do?

[deepgram-community[bot]]

Thanks for asking your question. Please be sure to reply with as much detail as possible so the community can assist you efficiently.
_{Consider joining our Discord community for more opportunity to engage with your fellow Deepgram users. You can earn points which can be redeemed for cool stuff by being active in our communities!}

[deepgram-community[bot]]

Hey there! It looks like you haven't connected your GitHub account to your Deepgram account. You can do this at https://community.deepgram.com - being verified through this process will allow our team to help you in a much more streamlined fashion.

[deepgram-community[bot]]

It looks like we're missing some important information to help debug your issue. Would you mind providing us with the following details in a reply?

• A request ID that triggered your error or issue.

[jpvajda]

@GabrielCNogueira could you provide me with a request ID so I can take a look in our logs?

Also could you copy / paste your entire SettingsConfiguration in this thread for us to check out?

[GabrielCNogueira]

@jpvajda Forgive but I was not able to figure out how to get the request ID from the javascript SDK.

Though upon further testing, it seems to be an issue with Cartesia, the voice returned when I set the provider to "cartesia" is similar but slightly different from "aura-asteria-en".

Here is my full configuration:

{
   "type":"SettingsConfiguration",
   "audio":{
      "input":{
         "encoding":"mulaw",
         "sample_rate":8000
      },
      "output":{
         "encoding":"mulaw",
         "sample_rate":8000,
         "container":"none"
      }
   },
   "agent":{
      "listen":{
         "model":"nova-2"
      },
      "think":{
         "provider":{
            "type":"open_ai"
         },
         "model":"gpt-4o-mini"
      },
      "speak":{
         "provider":"cartesia",
         "voice_id":""
      }
   }
}

I have a voice_id set, but I don't know if I can share it. Changing the ID to different voices also seems to have no effect.

[GabrielCNogueira]

@jpvajda I figured out the issue while playing with Cartesia's API:

If you run this curl, without "language" set, it generates the audio in Portuguese with an English accent:

curl -N -X POST "https://api.cartesia.ai/tts/bytes" \
        -H "Cartesia-Version: 2024-06-10" \
        -H "X-API-Key: " \
        -H "Content-Type: application/json" \
        -d '{"transcript": "Bem vindo ao Cartesia Sonic!", "model_id": "sonic", "voice": {"mode":"id", "id": "1cf751f6-8749-43ab-98bd-230dd633abdb"}, "output_format":{"container":"wav", "encoding":"pcm_f32le", "sample_rate":44100}}' > sonic.wav

Though if you add "language": "pt" to the payload, it comes as regular Portuguese:

curl -N -X POST "https://api.cartesia.ai/tts/bytes" \
       -H "Cartesia-Version: 2024-06-10" \
       -H "X-API-Key: " \
       -H "Content-Type: application/json" \
       -d '{"transcript": "Bem vindo ao Cartesia Sonic!", "model_id": "sonic", "voice": {"mode":"id", "id": "1cf751f6-8749-43ab-98bd-230dd633abdb"}, "output_format":{"container":"wav", "encoding":"pcm_f32le", "sample_rate":44100}, "language": "pt"}' > sonic.wav

Is there a way to set this in Deepgram? I tried adding "language": "pt" to the speak parameter but it is not valid.

[deepgram-community[bot]]

Thanks for the additional details. I'm curious if you can set that specific language option with Cartesia and then call that model with the Voice ID only from the SettingsConfig?

OR are you are not able to do that at all?

This message was sent by John Vajda from Deepgram, via our community automation.

[GabrielCNogueira]

@jpvajda I tried using both a default voice and a custom localized voice, in their platform both work fine.

But while calling the API, if the language is not specified, it outputs a voice with english accent. It's really weird.

They do not call the public API from their platform, so there is no clue whats going wrong.

Do you think it's something you guys can fix from your end or should I try contacting them?

[deepgram-community[bot]]

Just an update here: we are looking at adding support for a language option in the Voice Agent Settings Config to help with this problem. It's something that should be available in a future release of the API. I don't have exact details yet when that will be released.

This message was sent by John Vajda from Deepgram, via our community automation.