Clients
The LiveKit plugin
livekit-plugins-vaakini: install it, construct TTS, change settings mid-session, and what it reports.
For LiveKit Agents 1.8.5 or later, on Python 3.10 or later.
Install#
pip install livekit-plugins-vaakiniSet VAAKINI_API_KEY to your API key, or pass api_key.
Usage#
Construct TTS and hand it to your AgentSession.
from livekit.agents import AgentSessionfrom livekit.plugins import vaakini# The API key comes from VAAKINI_API_KEY.tts = vaakini.TTS( voice="Monika", language="hi",)session = AgentSession( tts=tts, # ... stt, llm, vad)TTS(...)#
Every argument is a keyword.
voicestringRequired- One of Vaakini’s voices by name, in any case (
Monika), or the id of one of your org’s custom voices (cv_…). Voices and languages lists them. languagestringOptional- The language the text is in:
as,bn,en,gu,hi,kn,ml,mr,or,pa,taorte. Left out, the model picks it from the script of each sentence. modelstringDefault "vaakini-tts-1"- The model. This is the only one.
normalizebooleanOptional; None: on- Speak numbers, currency, dates, times, OTPs, phone numbers, URLs and emails as words in the request’s language.
falseturns it off: the text is spoken as written. temperaturenumberOptional; None: the model's default- Sampling temperature, 0.1 to 1.5: lower is steadier, higher is more varied.
top_kintegerOptional; None: the model's default- Sample from the k likeliest codes, 1 to 2048.
top_pnumberOptional; None: the model's default- Nucleus sampling, above 0 and at most 1.
repetition_penaltynumberOptional; None: the model's default- Penalty on recently repeated codes, 0 to 5.
api_keystringDefault: $VAAKINI_API_KEY- Your Vaakini API key.
base_urlstringDefault wss://api.vaakini.com- Vaakini's URL.
http_sessionaiohttp.ClientSessionDefault: the job's- An aiohttp session to connect with.
The audio is 24000 Hz, 16-bit, mono. Vaakini checks the settings: when the session starts, the plugin connects and a wrong setting is logged as an error, and a reply with a wrong one fails with APIStatusError (bad_request, not retryable). Options says what each one does.
A whole agent#
Speech to text, an LLM and Vaakini, in one file.
from livekit.agents import Agent, AgentServer, AgentSession, JobContext, clifrom livekit.plugins import deepgram, openai, silero, vaakiniserver = AgentServer()@server.rtc_session()async def entrypoint(ctx: JobContext) -> None: session = AgentSession( vad=silero.VAD.load(), stt=deepgram.STT(model="nova-3", language="multi"), llm=openai.LLM(model="gpt-4.1-mini"), tts=vaakini.TTS(voice="Monika", language="hi"), ) agent = Agent( instructions=( "You are a receptionist at City Hospital. Speak Hindi in Devanagari script, and keep " "replies to two or three short sentences. No markdown, emojis or lists." ), ) await session.start(agent=agent, room=ctx.room) session.generate_reply(instructions="Greet the caller and ask how you can help.")if __name__ == "__main__": cli.run_app(server)It also needs livekit-agents[deepgram,openai,silero], with DEEPGRAM_API_KEY and OPENAI_API_KEY set. Talk to it in your terminal:
python agent.py consoleChanging settings#
update_options changes the voice, the language or an option mid-session.
tts.update_options(voice="Bunty", temperature=0.6)tts.update_options(temperature=None) # back to the model's defaultReplies that start afterwards use the new settings. A setting left out keeps its value; None puts an option back to the model’s default.
How a reply is spoken#
The plugin streams a reply’s text to Vaakini as the LLM writes it. Vaakini splits it into sentences and speaks each as soon as it is complete, so the first audio follows the reply’s first sentence. Transcripts follow the audio a sentence at a time.
tts.synthesize(text) speaks a text you have in full, the same way.
Connections#
When an AgentSession starts, it calls tts.prewarm(): the plugin connects to Vaakini and readies the model for the current settings, so the first reply doesn’t wait for either. Idle connections stay open between replies, for up to about 30 minutes, and are replaced when a Vaakini server restarts.
Interruptions#
When the caller interrupts, the plugin cancels the reply at Vaakini, which stops speaking it, and keeps the connection open for the next reply.
Metrics#
TTSMetrics.characters_count is the characters Vaakini spoke for the reply, which are the ones it charges for: the reply’s sentences, without the spaces between them. An interrupted reply reports its metrics too (TTSMetrics.cancelled set), counting the sentences sent in full before the interruption.
Errors#
Vaakini's errors become LiveKit's API errors, so LiveKit's retries and FallbackAdapter treat them as any TTS's.
| What happened | Raised |
|---|---|
| An error from Vaakini during a reply | APIStatusError, with the error’s retryable and the status its code has over HTTP |
| Vaakini refused the API key | APIStatusError, status 401 |
| Your org is out of credit | APIStatusError, status 402 |
| The connection failed or dropped | APIConnectionError |
| Vaakini didn't connect or sent nothing in time | APITimeoutError |
| No API key, at construction | ValueError |