Vaakini docs

Clients

The LiveKit plugin

livekit-plugins-vaakini: install it, construct TTS, change settings mid-session, and what it reports.

View as Markdown

For LiveKit Agents 1.8.5 or later, on Python 3.10 or later.

Install#

pip install livekit-plugins-vaakini

Set VAAKINI_API_KEY to your API key, or pass api_key.

Usage#

Construct TTS and hand it to your AgentSession.

Python
from livekit.agents import AgentSessionfrom livekit.plugins import vaakini# The API key comes from VAAKINI_API_KEY.tts = vaakini.TTS(    voice="Monika",    language="hi",)session = AgentSession(    tts=tts,    # ... stt, llm, vad)

TTS(...)#

Every argument is a keyword.

voicestringRequired
One of Vaakini’s voices by name, in any case (Monika), or the id of one of your org’s custom voices (cv_…). Voices and languages lists them.
languagestringOptional
The language the text is in: as, bn, en, gu, hi, kn, ml, mr, or, pa, ta or te. Left out, the model picks it from the script of each sentence.
modelstringDefault "vaakini-tts-1"
The model. This is the only one.
normalizebooleanOptional; None: on
Speak numbers, currency, dates, times, OTPs, phone numbers, URLs and emails as words in the request’s language. false turns it off: the text is spoken as written.
temperaturenumberOptional; None: the model's default
Sampling temperature, 0.1 to 1.5: lower is steadier, higher is more varied.
top_kintegerOptional; None: the model's default
Sample from the k likeliest codes, 1 to 2048.
top_pnumberOptional; None: the model's default
Nucleus sampling, above 0 and at most 1.
repetition_penaltynumberOptional; None: the model's default
Penalty on recently repeated codes, 0 to 5.
api_keystringDefault: $VAAKINI_API_KEY
Your Vaakini API key.
base_urlstringDefault wss://api.vaakini.com
Vaakini's URL.
http_sessionaiohttp.ClientSessionDefault: the job's
An aiohttp session to connect with.

The audio is 24000 Hz, 16-bit, mono. Vaakini checks the settings: when the session starts, the plugin connects and a wrong setting is logged as an error, and a reply with a wrong one fails with APIStatusError (bad_request, not retryable). Options says what each one does.

A whole agent#

Speech to text, an LLM and Vaakini, in one file.

agent.py
from livekit.agents import Agent, AgentServer, AgentSession, JobContext, clifrom livekit.plugins import deepgram, openai, silero, vaakiniserver = AgentServer()@server.rtc_session()async def entrypoint(ctx: JobContext) -> None:    session = AgentSession(        vad=silero.VAD.load(),        stt=deepgram.STT(model="nova-3", language="multi"),        llm=openai.LLM(model="gpt-4.1-mini"),        tts=vaakini.TTS(voice="Monika", language="hi"),    )    agent = Agent(        instructions=(            "You are a receptionist at City Hospital. Speak Hindi in Devanagari script, and keep "            "replies to two or three short sentences. No markdown, emojis or lists."        ),    )    await session.start(agent=agent, room=ctx.room)    session.generate_reply(instructions="Greet the caller and ask how you can help.")if __name__ == "__main__":    cli.run_app(server)

It also needs livekit-agents[deepgram,openai,silero], with DEEPGRAM_API_KEY and OPENAI_API_KEY set. Talk to it in your terminal:

python agent.py console

Changing settings#

update_options changes the voice, the language or an option mid-session.

Python
tts.update_options(voice="Bunty", temperature=0.6)tts.update_options(temperature=None)  # back to the model's default

Replies that start afterwards use the new settings. A setting left out keeps its value; None puts an option back to the model’s default.

How a reply is spoken#

The plugin streams a reply’s text to Vaakini as the LLM writes it. Vaakini splits it into sentences and speaks each as soon as it is complete, so the first audio follows the reply’s first sentence. Transcripts follow the audio a sentence at a time.

tts.synthesize(text) speaks a text you have in full, the same way.

Connections#

When an AgentSession starts, it calls tts.prewarm(): the plugin connects to Vaakini and readies the model for the current settings, so the first reply doesn’t wait for either. Idle connections stay open between replies, for up to about 30 minutes, and are replaced when a Vaakini server restarts.

Interruptions#

When the caller interrupts, the plugin cancels the reply at Vaakini, which stops speaking it, and keeps the connection open for the next reply.

Metrics#

TTSMetrics.characters_count is the characters Vaakini spoke for the reply, which are the ones it charges for: the reply’s sentences, without the spaces between them. An interrupted reply reports its metrics too (TTSMetrics.cancelled set), counting the sentences sent in full before the interruption.

Errors#

Vaakini's errors become LiveKit's API errors, so LiveKit's retries and FallbackAdapter treat them as any TTS's.

What happenedRaised
An error from Vaakini during a replyAPIStatusError, with the error’s retryable and the status its code has over HTTP
Vaakini refused the API keyAPIStatusError, status 401
Your org is out of creditAPIStatusError, status 402
The connection failed or droppedAPIConnectionError
Vaakini didn't connect or sent nothing in timeAPITimeoutError
No API key, at constructionValueError