> ## Documentation Index > Fetch the complete documentation index at: https://docs.chariot.in/llms.txt > Use this file to discover all available pages before exploring further. # Multi-context WebSocket > Run several independent speech streams over one WebSocket, and interrupt any of them instantly. Built for voice agents. > [!WARNING] > > **Beta.** The multi-context WebSocket is in beta. The message protocol may change based on feedback before general availability. The [WebSocket](/guides/text-to-speech/websocket) at `/v1/tts/ws` remains fully supported. The multi-context WebSocket carries several independent streams of speech, called **contexts**, over a single connection. Each context has its own text buffer, voice, and lifecycle. You can open a new context for the next reply, or cancel the one that is speaking, without reconnecting. It is designed for **voice agents**: when a user interrupts, cancel the current reply and start the next one on the same socket. ## When to use it | Use `/v1/tts/ws` when | Use `/v1/tts/ws-multi` when | | - | - | | You generate one stream of speech at a time. | You need to interrupt speech mid-sentence (barge-in) and keep the connection. | | You never need to stop audio that is already generating. | You want to prepare the next reply while the current one is still speaking. | | You want raw binary audio frames. | You want two or more replies in flight at once, for example a filler line and the real answer. | | You use one voice for the whole session. | You want a different voice or speed for some turns without reconnecting. | For help choosing between REST, HTTP streaming, and the WebSockets, see [Choosing an API](/guides/text-to-speech/which-api). ## Endpoint ``` wss://api.chariot.in/v1/tts/ws-multi ``` For the full message schemas, see the [Multi-context WebSocket reference](/api-reference/endpoint/tts-websocket-multi). ## Authenticating Send your API key in the `chariotai-api-key` header. Browser clients that cannot set handshake headers can pass it as a query parameter instead: ``` wss://api.chariot.in/v1/tts/ws-multi?voice_id=VOICE_ID&api_key=YOUR_API_KEY ``` If authentication fails, the server accepts the socket, sends an `error` frame, and closes with code `1008`. ## Concepts ### Contexts A **context** is one stream of speech, usually one turn of a conversation. You name it with a `context_id` of your choice, and every message in both directions carries it. * The first `input.text` with a new `context_id` opens the context. There is no separate "create" message. * Each context has its own voice and speed, set on its first message or taken from the connection defaults. * Up to **5** contexts can be open on one connection at the same time. * A context ends with exactly one `context.done` message. Nothing is sent for that context after it. ### Segments Within a context, each `input.flush` turns the text buffered so far into one **segment** and generates it. Segments are numbered from `0` within each context and generated in order. All audio for a segment is tagged with its `segment` number, between an `audio.start` and an `audio.done`. ## How a context works **1. Send text** Send `input.text` with a `context_id`. The first message opens the context. Text accumulates in that context's buffer. Nothing is generated yet. **2. Flush to generate** Send `input.flush` for the context. Its buffered text is generated as the next segment and streamed back. The context stays open for more text. **3. Close or cancel** Send `context.close` when the reply is complete. The rest of the buffer is generated, all audio is delivered, and you receive `context.done` with reason `completed`. To stop immediately instead, send `context.cancel`. > [!NOTE] > > Flushing is what triggers generation. Flush at natural boundaries, such as the end of each sentence, so audio starts early and sounds natural. ## Interruptions (barge-in) When your user starts speaking over the agent, cancel the context that is speaking and start the next reply in a new context: ```json theme={null} { "type": "context.cancel", "context_id": "turn-1" } { "type": "input.text", "context_id": "turn-2", "text": "Sure, which account? " } { "type": "input.flush", "context_id": "turn-2" } ``` `context.cancel` stops the context at once: * Generation in progress for the context is stopped. * Its queued segments and unflushed text are discarded. * The server replies with `context.done` and reason `cancelled`. **No audio for that context is sent after it.** Audio that arrives between your cancel and its `context.done` was already in flight, so discard it. > [!WARNING] > > The server cannot recall audio it has already delivered. When you cancel, also stop playback and clear any audio for that context that is buffered in your player. > [!TIP] > > Use a new `context_id` for every turn, for example `turn-1`, `turn-2`. A finished `context_id` can be reused after its `context.done`, and it then starts a fresh context, but a new id per turn keeps late messages from an old turn easy to tell apart. ## Connection parameters Pass these as query parameters on the connection URL: **voice\_id** `string` UUID of the default voice. Used by every context that does not name its own `voice_id`. Optional: without it, each context must set `voice_id` on its first message. **model\_type** `string` (default: chariot-v0) Use `chariot-v0` or `chariot-multilingual-v1`. Fixed for the connection. Every voice used must support this model. See [Models](/guides/models). **speed** `number` (default: 1.0) Default speaking rate for contexts. Choose a value from the voice's `speeds` list for the model. Chariot v0 supports only `1.0`. **optimize\_streaming\_latency** `integer` (default: 0) An integer from `0` to `4`. Higher values trade quality passes for lower latency. Applies to the whole connection. **idle\_timeout** `number` (default: 60) Seconds the server waits for any client message before closing the connection as idle. Must be greater than 0 and at most 300. **enable\_logging** `boolean` (default: false) Controls whether Chariot stores the generated audio. When `true`, the audio for each generated segment is stored and you can retrieve it from your TTS history. ## Message protocol Every message, including audio, is a JSON text frame. ### Client to server **input.text** `message` Append text to a context's buffer. The first `input.text` for a `context_id` opens the context, and only that message may set `voice_id` and `speed`. An empty `text` sends nothing for generation and keeps the context alive. ```json theme={null} { "type": "input.text", "context_id": "turn-1", "text": "Your order shipped on Monday. ", "voice_id": "VOICE_ID" } ``` **input.flush** `message` Generate the context's buffered text as the next segment. The context stays open. ```json theme={null} { "type": "input.flush", "context_id": "turn-1" } ``` **context.close** `message` No more text is coming for this context. The rest of its buffer is generated, all audio is delivered, and then `context.done` arrives with reason `completed`. ```json theme={null} { "type": "context.close", "context_id": "turn-1" } ``` **context.cancel** `message` End the context now. See [Interruptions](#interruptions-barge-in). Cancelling a context that has already ended is ignored. ```json theme={null} { "type": "context.cancel", "context_id": "turn-1" } ``` **session.close** `message` Finish every open context as if each received `context.close`, then close the connection. ```json theme={null} { "type": "session.close" } ``` **ping** `message` Keep the connection alive. The server answers with `pong`. ```json theme={null} { "type": "ping" } ``` `context_id` is 1 to 128 characters of letters, digits, `_`, `-`, `.`, and `:`. If you omit it, the message applies to a context named `default`. ### Server to client **session.ready** `message` The connection is authenticated and ready. ```json theme={null} { "type": "session.ready", "session_id": "3fa92f3b-6b45-427e-8f4c-e33f5087e136", "max_contexts": 5, "sample_rate": 44100, "encoding": "pcm_s16le" } ``` **audio.start** `message` A segment of a context is about to stream. ```json theme={null} { "type": "audio.start", "context_id": "turn-1", "segment": 0, "sample_rate": 44100 } ``` **audio.chunk** `message` Audio for a segment: base64-encoded raw PCM, signed 16-bit little-endian, mono. ```json theme={null} { "type": "audio.chunk", "context_id": "turn-1", "segment": 0, "audio": "AAD/////AQACAAMA..." } ``` **audio.done** `message` The segment has finished streaming. `credits_utilized` is what this segment cost. ```json theme={null} { "type": "audio.done", "context_id": "turn-1", "segment": 0, "credits_utilized": 31, "error": false } ``` **context.done** `message` The context has ended. This is always the last message for a context. Carries the `reason`, the number of completed `segments`, and the total `credits_utilized`. When `reason` is `error`, it also carries a `code` and `message`. ```json theme={null} { "type": "context.done", "context_id": "turn-1", "reason": "completed", "segments": 2, "credits_utilized": 58 } ``` **error** `message` A message was rejected. Carries a `code`, a `message`, and the `context_id` it concerns, if any. The connection stays open unless the error is `insufficient_credits` or `idle_timeout`. ```json theme={null} { "type": "error", "code": "context_limit", "message": "5 contexts are already open; close or cancel one first", "context_id": "turn-6" } ``` **session.done** `message` Sent after `session.close` once every context has finished. The server then closes the connection with code `1000`. ```json theme={null} { "type": "session.done", "contexts": 3, "credits_utilized": 136 } ``` ### Why a context ended | `reason` | Meaning | | - | - | | `completed` | You sent `context.close` and all audio was delivered. | | `cancelled` | You sent `context.cancel`. | | `expired` | The context was idle for 20 seconds (see [Keeping contexts alive](#keeping-contexts-alive)). | | `error` | The context could not continue. See `code` and `message`. | | `session_closed` | The connection ended while the context was still open. | ### Error codes Errors in an `error` message, or in a `context.done` with reason `error`. Only `insufficient_credits` and `idle_timeout` close the connection; every other error affects one message or one context. | `code` | Meaning | | - | - | | `bad_request` | The message is malformed, has an unknown `type`, or tries to change `voice_id` or `speed` after the context's first message. | | `unknown_context` | `input.flush` or `context.close` names a context that is not open. | | `context_limit` | 5 contexts are already open. The new context was not opened. | | `context_closing` | Text was sent to a context after its `context.close`. | | `session_closing` | A new context was opened after `session.close`. | | `text_too_long` | One `input.text` is longer than the model limit. The fragment was rejected. | | `voice_unavailable` | The voice does not exist, is not available to your workspace, or is not ready. | | `concurrency_limit` | Your workspace reached its concurrent generation limit. Only this context ended. | | `forbidden` | Your plan does not include text to speech. | | `upstream_error` | Generation failed for this context. Other contexts are unaffected. | | `insufficient_credits` | Your workspace cannot pay for the text. The connection closes with `4402`. | | `idle_timeout` | No client message within `idle_timeout`. The connection closes with `1008`. | ## Keeping contexts alive Two timers apply: * **Context idle timeout, 20 seconds.** A context with nothing generating and no message for 20 seconds ends with reason `expired`, and its unflushed text is discarded. To keep a context open while your agent waits, for example on a slow tool call, send an empty `input.text` for it: ```json theme={null} { "type": "input.text", "context_id": "turn-1", "text": "" } ``` * **Connection idle timeout.** If no message of any kind arrives within `idle_timeout` (default 60 seconds), the connection closes with code `1008`. Send `ping` while your client is listening but not sending. ## Audio format Audio arrives in `audio.chunk` messages as base64-encoded raw PCM: signed 16-bit little-endian, mono, at the `sample_rate` given in `audio.start` (44100 Hz). The `encoding` field of `session.ready` names the format: `pcm_s16le`. ## Close codes | Code | Meaning | | - | - | | `1000` | Normal, after `session.close` or when you close the connection. | | `1008` | Policy violation: authentication failed, an invalid parameter or default voice, or the idle timeout fired. | | `1011` | Internal error. | | `4402` | Insufficient credits. | ## Billing Billing is **per segment**, at **1 credit per input character**, exactly as on `/v1/tts/ws`. Each segment is charged when it finishes, and `audio.done` reports its cost. `context.done` reports the total for the context. When you cancel a context: * A segment that had **started generating** is charged in full. * Segments that were still queued, and text that was never flushed, are **not charged**. The server checks your available credits before accepting each `input.text`, across all open contexts. If your credits cannot cover the pending text, you receive `insufficient_credits` and the connection closes with `4402`. ## Concurrency Each segment that is generating uses one slot of your workspace's [concurrency limit](/guides/resources/rate-limits), the same limit shared with the HTTP endpoints and `/v1/tts/ws`. Open contexts that are not generating use no slot. If a segment is refused a slot, only its context ends, with reason `error` and code `concurrency_limit`. The connection and your other contexts continue. ## Limits | Limit | Value | | - | - | | Open contexts per connection | 5 | | Context idle timeout | 20 seconds | | Connection idle timeout | Configurable, default 60 s, maximum 300 s | | Each `input.text` fragment | 500 characters for `chariot-v0`; 1,000 for `chariot-multilingual-v1` | | Unflushed text per context | The same model limit. Text that would exceed it is generated as a segment first, so no text is dropped. | ## Example A two-turn conversation where the user interrupts the first reply. Replace `VOICE_ID` with a voice UUID that supports `chariot-multilingual-v1`. ```python Python theme={null} import asyncio import base64 import json import websockets URL = ( "wss://api.chariot.in/v1/tts/ws-multi" "?voice_id=VOICE_ID&model_type=chariot-multilingual-v1" ) HEADERS = {"chariotai-api-key": "YOUR_API_KEY"} async def main(): async with websockets.connect(URL, additional_headers=HEADERS) as ws: cancelled = set() async def send(msg): await ws.send(json.dumps(msg)) async def receive(): async for raw in ws: event = json.loads(raw) kind, ctx = event["type"], event.get("context_id") if kind == "audio.chunk": if ctx not in cancelled: # drop audio still in flight after a cancel play_pcm(ctx, base64.b64decode(event["audio"])) # 44.1 kHz, mono, s16le elif kind == "context.done": print(ctx, "ended:", event["reason"], "credits:", event["credits_utilized"]) elif kind == "error": print("error:", event["code"], event["message"]) elif kind == "session.done": return receiver = asyncio.create_task(receive()) # Turn 1: stream the reply, flushing at the end of each sentence. await send({"type": "input.text", "context_id": "turn-1", "text": "Your order shipped on Monday. "}) await send({"type": "input.flush", "context_id": "turn-1"}) await send({"type": "input.text", "context_id": "turn-1", "text": "It should arrive by Thursday evening. "}) await send({"type": "input.flush", "context_id": "turn-1"}) await asyncio.sleep(1.0) # the user starts speaking # Barge-in: stop turn 1 (and its playback), answer in turn 2 on the same socket. cancelled.add("turn-1") stop_playback("turn-1") await send({"type": "context.cancel", "context_id": "turn-1"}) await send({"type": "input.text", "context_id": "turn-2", "text": "Sure, which account? "}) await send({"type": "context.close", "context_id": "turn-2"}) # Finish once every context is done. await send({"type": "session.close"}) await receiver asyncio.run(main()) ``` This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.