# The Pipecat service

> pipecat-vaakini: install it, construct VaakiniTTSService, change settings mid-session, and what it reports.

For [Pipecat](https://github.com/pipecat-ai/pipecat) 1.12.0 or later, on Python 3.11 or later. Tested with Pipecat 1.12.0 and 1.16.0.

## Install

```shell
pip install pipecat-vaakini
```

## Usage

Construct the service and put it where a TTS service goes, after the LLM.

```python
import os

from pipecat_vaakini import VaakiniTTSService

tts = VaakiniTTSService(
    api_key=os.environ["VAAKINI_API_KEY"],
    settings=VaakiniTTSService.Settings(
        voice="Monika",
        language="hi",
    ),
)

# In your pipeline, where a TTS service goes, after the LLM:
# Pipeline([transport.input(), stt, user_aggregator, llm, tts, transport.output(), assistant_aggregator])
```

## VaakiniTTSService(...)

- `api_key` (`string`, Required): Your Vaakini API key.
- `settings` (`VaakiniTTSService.Settings`, Required): The voice and how it speaks, below.
- `base_url` (`string`, Default wss://api.vaakini.com): Vaakini's URL.

Other keyword arguments go to Pipecat’s `WebsocketTTSService`, except two. `text_aggregation_mode` is refused: Vaakini splits the text into sentences itself, so the service streams the LLM’s text to it as it arrives. `sample_rate` is refused too: the audio is 24000 Hz, and the pipeline’s output transport resamples it to its own rate.

## Settings

`VaakiniTTSService.Settings(...)`, which is `VaakiniTTSSettings`.

- `voice` (`string`, Required): One of Vaakini’s voices by name, in any case (`Monika`), or the id of one of your org’s [custom voices](https://docs.vaakini.com/custom-voices.md) (`cv_…`). [Voices and languages](https://docs.vaakini.com/voices.md) lists them.
- `language` (`string`, Optional): The language the text is in: `as` , `bn` , `en` , `gu` , `hi` , `kn` , `ml` , `mr` , `or` , `pa` , `ta` or `te`. Left out, the model picks it from the script of each sentence. A Pipecat `Language` works too (`Language.HI`; a region, as in `Language.HI_IN`, is dropped).
- `model` (`string`, Default "vaakini-tts-1"): The model. This is the only one.
- `normalize` (`boolean`, Optional; None: on): Speak numbers, currency, dates, times, OTPs, phone numbers, URLs and emails as words in the request’s language. `false` turns it off: the text is spoken as written.
- `temperature` (`number`, Optional; None: the model's default): Sampling temperature, 0.1 to 1.5: lower is steadier, higher is more varied.
- `top_k` (`integer`, Optional; None: the model's default): Sample from the k likeliest codes, 1 to 2048.
- `top_p` (`number`, Optional; None: the model's default): Nucleus sampling, above 0 and at most 1.
- `repetition_penalty` (`number`, Optional; None: the model's default): Penalty on recently repeated codes, 0 to 5.

Vaakini checks the settings: a wrong one is reported as an `ErrorFrame`. [Options](https://docs.vaakini.com/options.md) says what each one does.

## A whole bot

Speech to text, an LLM and Vaakini, on Pipecat's runner.

**bot.py**

```python
import os

from pipecat.audio.vad.silero import SileroVADAnalyzer
from pipecat.frames.frames import TTSSpeakFrame
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.worker import PipelineParams, PipelineWorker
from pipecat.processors.aggregators.llm_context import LLMContext
from pipecat.processors.aggregators.llm_response_universal import (
    LLMContextAggregatorPair,
    LLMUserAggregatorParams,
)
from pipecat.runner.types import RunnerArguments
from pipecat.runner.utils import create_transport
from pipecat.services.deepgram.stt import DeepgramSTTService
from pipecat.services.openai.llm import OpenAILLMService
from pipecat.transports.base_transport import BaseTransport, TransportParams
from pipecat.workers.runner import WorkerRunner
from pipecat_vaakini import VaakiniTTSService

transport_params = {
    "webrtc": lambda: TransportParams(audio_in_enabled=True, audio_out_enabled=True),
}


async def run_bot(transport: BaseTransport, runner_args: RunnerArguments):
    stt = DeepgramSTTService(
        api_key=os.environ["DEEPGRAM_API_KEY"],
        settings=DeepgramSTTService.Settings(model="nova-3", language="multi"),
    )
    llm = OpenAILLMService(
        api_key=os.environ["OPENAI_API_KEY"],
        settings=OpenAILLMService.Settings(
            model="gpt-4.1-mini",
            system_instruction=(
                "You are a receptionist at City Hospital. Speak Hindi in Devanagari script, and "
                "keep replies to two or three short sentences. No markdown, emojis or lists."
            ),
        ),
    )
    tts = VaakiniTTSService(
        api_key=os.environ["VAAKINI_API_KEY"],
        settings=VaakiniTTSService.Settings(voice="Monika", language="hi"),
    )

    context = LLMContext()
    user_aggregator, assistant_aggregator = LLMContextAggregatorPair(
        context,
        user_params=LLMUserAggregatorParams(vad_analyzer=SileroVADAnalyzer()),
    )
    pipeline = Pipeline(
        [transport.input(), stt, user_aggregator, llm, tts, transport.output(), assistant_aggregator]
    )
    worker = PipelineWorker(
        pipeline,
        params=PipelineParams(enable_metrics=True, enable_usage_metrics=True),
        idle_timeout_secs=runner_args.pipeline_idle_timeout_secs,
    )
    runner = WorkerRunner(handle_sigint=runner_args.handle_sigint)
    await runner.add_workers(worker)

    @transport.event_handler("on_client_connected")
    async def on_client_connected(transport, client):
        await worker.queue_frames([TTSSpeakFrame("नमस्ते, मैं सिटी हॉस्पिटल से बोल रही हूँ। मैं आपकी क्या मदद कर सकती हूँ?")])

    @transport.event_handler("on_client_disconnected")
    async def on_client_disconnected(transport, client):
        await runner.cancel()

    await runner.run()


async def bot(runner_args: RunnerArguments):
    transport = await create_transport(runner_args, transport_params)
    await run_bot(transport, runner_args)


if __name__ == "__main__":
    from pipecat.runner.run import main

    main()
```

It also needs `pipecat-ai[deepgram,openai,runner,silero,webrtc]`, with `DEEPGRAM_API_KEY` and `OPENAI_API_KEY` set. Run it, then open `http://localhost:7860/client` and talk to it:

```shell
python bot.py
```

## Changing settings

A TTSUpdateSettingsFrame changes the voice, the language or an option mid-session.

```python
from pipecat.frames.frames import TTSUpdateSettingsFrame

await worker.queue_frame(
    TTSUpdateSettingsFrame(delta=VaakiniTTSService.Settings(voice="Bunty", temperature=0.6))
)
```

Replies that start afterwards use the new settings. A setting left out of the update keeps its value; `None` puts an option back to the model’s default (`Settings(temperature=None)`).

## How a reply is spoken

The service streams the LLM’s text to Vaakini as it arrives. Vaakini splits it into sentences and speaks each as soon as it is complete, so the first audio follows the reply’s first sentence.

## Interruptions

When the user interrupts, the service tells Vaakini to stop the reply. The next reply starts on the same connection.

## Transcripts

The service’s `TTSTextFrame`s follow the audio: each sentence’s words as its audio starts.

## Metrics

With usage metrics on (`enable_usage_metrics=True`), each reply’s `TTSUsageMetricsData` is the characters Vaakini spoke, which are the ones it charges for: the reply’s sentences, without the spaces between them. An interrupted reply counts the sentences sent in full before the interruption.

## Errors

Vaakini's errors become Pipecat ErrorFrames, each with the ErrorCategory that matches its code.

| code | ErrorCategory |
| --- | --- |
| bad_request | `INVALID_REQUEST` |
| unauthorized | `AUTHENTICATION` |
| insufficient_credits | `QUOTA` |
| rate_limited | `RATE_LIMIT` |
| unavailable, timeout, internal | `SERVER` |

A connection that fails or drops is an `ErrorFrame` with `CONNECTIVITY`. [Errors](https://docs.vaakini.com/errors.md) says what each code means.
