# The LiveKit plugin

> livekit-plugins-vaakini: install it, construct TTS, change settings mid-session, and what it reports.

For [LiveKit Agents](https://github.com/livekit/agents) 1.8.5 or later, on Python 3.10 or later.

## Install

```shell
pip install livekit-plugins-vaakini
```

Set `VAAKINI_API_KEY` to your API key, or pass `api_key`.

## Usage

Construct TTS and hand it to your AgentSession.

```python
from livekit.agents import AgentSession
from livekit.plugins import vaakini

# The API key comes from VAAKINI_API_KEY.
tts = vaakini.TTS(
    voice="Monika",
    language="hi",
)

session = AgentSession(
    tts=tts,
    # ... stt, llm, vad
)
```

## TTS(...)

Every argument is a keyword.

- `voice` (`string`, Required): One of Vaakini’s voices by name, in any case (`Monika`), or the id of one of your org’s [custom voices](https://docs.vaakini.com/custom-voices.md) (`cv_…`). [Voices and languages](https://docs.vaakini.com/voices.md) lists them.
- `language` (`string`, Optional): The language the text is in: `as` , `bn` , `en` , `gu` , `hi` , `kn` , `ml` , `mr` , `or` , `pa` , `ta` or `te`. Left out, the model picks it from the script of each sentence.
- `model` (`string`, Default "vaakini-tts-1"): The model. This is the only one.
- `normalize` (`boolean`, Optional; None: on): Speak numbers, currency, dates, times, OTPs, phone numbers, URLs and emails as words in the request’s language. `false` turns it off: the text is spoken as written.
- `temperature` (`number`, Optional; None: the model's default): Sampling temperature, 0.1 to 1.5: lower is steadier, higher is more varied.
- `top_k` (`integer`, Optional; None: the model's default): Sample from the k likeliest codes, 1 to 2048.
- `top_p` (`number`, Optional; None: the model's default): Nucleus sampling, above 0 and at most 1.
- `repetition_penalty` (`number`, Optional; None: the model's default): Penalty on recently repeated codes, 0 to 5.
- `api_key` (`string`, Default: $VAAKINI_API_KEY): Your Vaakini API key.
- `base_url` (`string`, Default wss://api.vaakini.com): Vaakini's URL.
- `http_session` (`aiohttp.ClientSession`, Default: the job's): An aiohttp session to connect with.

The audio is 24000 Hz, 16-bit, mono. Vaakini checks the settings: when the session starts, the plugin connects and a wrong setting is logged as an error, and a reply with a wrong one fails with `APIStatusError` (`bad_request`, not retryable). [Options](https://docs.vaakini.com/options.md) says what each one does.

## A whole agent

Speech to text, an LLM and Vaakini, in one file.

**agent.py**

```python
from livekit.agents import Agent, AgentServer, AgentSession, JobContext, cli
from livekit.plugins import deepgram, openai, silero, vaakini

server = AgentServer()


@server.rtc_session()
async def entrypoint(ctx: JobContext) -> None:
    session = AgentSession(
        vad=silero.VAD.load(),
        stt=deepgram.STT(model="nova-3", language="multi"),
        llm=openai.LLM(model="gpt-4.1-mini"),
        tts=vaakini.TTS(voice="Monika", language="hi"),
    )
    agent = Agent(
        instructions=(
            "You are a receptionist at City Hospital. Speak Hindi in Devanagari script, and keep "
            "replies to two or three short sentences. No markdown, emojis or lists."
        ),
    )
    await session.start(agent=agent, room=ctx.room)
    session.generate_reply(instructions="Greet the caller and ask how you can help.")


if __name__ == "__main__":
    cli.run_app(server)
```

It also needs `livekit-agents[deepgram,openai,silero]`, with `DEEPGRAM_API_KEY` and `OPENAI_API_KEY` set. Talk to it in your terminal:

```shell
python agent.py console
```

## Changing settings

update_options changes the voice, the language or an option mid-session.

```python
tts.update_options(voice="Bunty", temperature=0.6)
tts.update_options(temperature=None)  # back to the model's default
```

Replies that start afterwards use the new settings. A setting left out keeps its value; `None` puts an option back to the model’s default.

## How a reply is spoken

The plugin streams a reply’s text to Vaakini as the LLM writes it. Vaakini splits it into sentences and speaks each as soon as it is complete, so the first audio follows the reply’s first sentence. Transcripts follow the audio a sentence at a time.

`tts.synthesize(text)` speaks a text you have in full, the same way.

## Connections

When an `AgentSession` starts, it calls `tts.prewarm()`: the plugin connects to Vaakini and readies the model for the current settings, so the first reply doesn’t wait for either. Idle connections stay open between replies, for up to about 30 minutes, and are replaced when a Vaakini server restarts.

## Interruptions

When the caller interrupts, the plugin cancels the reply at Vaakini, which stops speaking it, and keeps the connection open for the next reply.

## Metrics

`TTSMetrics.characters_count` is the characters Vaakini spoke for the reply, which are the ones it charges for: the reply’s sentences, without the spaces between them. An interrupted reply reports its metrics too (`TTSMetrics.cancelled` set), counting the sentences sent in full before the interruption.

## Errors

Vaakini's errors become LiveKit's API errors, so LiveKit's retries and FallbackAdapter treat them as any TTS's.

| What happened | Raised |
| --- | --- |
| An [error](https://docs.vaakini.com/errors.md) from Vaakini during a reply | `APIStatusError`, with the error’s `retryable` and the status its code has over HTTP |
| Vaakini refused the API key | `APIStatusError`, status 401 |
| Your org is out of credit | `APIStatusError`, status 402 |
| The connection failed or dropped | `APIConnectionError` |
| Vaakini didn't connect or sent nothing in time | `APITimeoutError` |
| No API key, at construction | `ValueError` |
