> ## Documentation Index > Fetch the complete documentation index at: https://docs.chariot.in/llms.txt > Use this file to discover all available pages before exploring further. # Latency testing > Measure first-audio latency and connection timings for Chariot HTTP streaming and WebSocket from your own environment. Measure how quickly you receive the first audio from Chariot using the Python latency tool. You can test HTTP streaming and WebSocket, compare fresh and reused connections, and save the results for inspection. > **[Get the latency tool](https://github.com/Chariot-AI/chariot-integrations/tree/main/tools/latency)** > > Read the source, install the dependencies, and run the same measurements from your laptop or application host. ## Set up You need Git, Python 3.10 or later, a Chariot API key, and a voice that supports your selected model. **1. Install the tool** ```bash theme={null} git clone https://github.com/Chariot-AI/chariot-integrations.git cd chariot-integrations/tools/latency python3 -m venv .venv source .venv/bin/activate pip install -r requirements.txt cp .env.sample .env ``` Run these commands in a macOS or Linux shell. Keep your terminal in `tools/latency` for the commands below. If you already have the repository, update your checkout from `main` first. **2. Configure your key, voice, and model** Edit `.env`: ```dotenv theme={null} CHARIOT_API_KEY=your-api-key CHARIOT_VOICE_ID=your-voice-id CHARIOT_MODEL=chariot-multilingual-v1 ``` Create your API key in the [Dashboard](https://platform.chariot.in). Get a voice UUID from [`GET /v1/voices`](/api-reference/endpoint/get-voices). Choose a voice whose `models[].model` includes `chariot-multilingual-v1`. The tool defaults to Chariot Multilingual v1 and sends `model_type` explicitly. You can override `CHARIOT_MODEL` with `--model`. See [Models](/guides/models) for compatible voices, input limits, and speaking rates. **3. Run the latency test** ```bash theme={null} python latency.py --model chariot-multilingual-v1 --json results.json ``` You receive per-request timings, connection measurements, and a TTFB summary with min, p50, p90, p99, and max. The JSON file preserves the individual measurements and configuration. > [!NOTE] > > The default run makes **20 sequential synthesis requests**. Normal [API credits](/guides/resources/credits) apply. There are no hidden warm-up requests or automatic retries. ## Choose what to test Both streaming endpoints are included by default: | CLI transport | Endpoint | Default requests | | - | - | - | | `http` | [`POST /v1/tts/stream`](/api-reference/endpoint/tts-stream) | 5 fresh, then 5 on a shared connection | | `websocket` | [`wss://api.chariot.in/v1/tts/ws`](/guides/text-to-speech/websocket) | 5 fresh, then 5 on a shared session | The tool sends the same complete text for every request. It measures latency rather than concurrency, throughput, real-time factor, or playback underruns. ```bash HTTP streaming theme={null} python latency.py --transport http --connection both --runs 5 \ --json http-results.json ``` ```bash WebSocket theme={null} python latency.py --transport websocket --connection both --runs 5 \ --json websocket-results.json ``` ```bash Reuse a WebSocket theme={null} python latency.py --transport websocket --connection reused --runs 5 \ --json reused-results.json ``` ```bash Save audio theme={null} python latency.py --text "नमस्ते, आपका दिन कैसा चल रहा है?" \ --model chariot-multilingual-v1 --save-audio audio --json results.json ``` `--save-audio` writes one WAV file per successful request after consuming the response. File writes do not enter the first-audio timing. Reusing an output path overwrites its previous results. ### Fresh and reused connections * **Fresh:** each request opens a new connection. DNS caches and the model may already be warm. * **Reused HTTP:** the tool reads the complete response before sending the next request on the same HTTP/1.1 connection. * **Reused WebSocket:** the tool sends `input.text` followed by `input.flush`, reads through `audio.done`, and repeats on the same session. It sends `input.done` when finished. With `--runs 5`, each reused group contains one request labeled `first` and four labeled `reused`. The summary keeps them separate. With `--runs 1`, you get no reused samples. ## Read the results The tool prints endpoint URLs before the trials. In the output, **`http` means HTTP streaming**. This illustrative excerpt shows the output format, not expected latency: ```text theme={null} http fresh 1 fresh setup= 30.0 ms TTFB= 150.0 ms setup+TTFB= 180.0 ms websocket reused 1 first setup= 100.0 ms TTFB= 180.0 ms setup+TTFB= 280.0 ms websocket reused 2 reused setup= 0.0 ms TTFB= 85.0 ms setup+TTFB= 85.0 ms websocket reused 3 reused setup= 0.0 ms TTFB= 90.0 ms setup+TTFB= 90.0 ms websocket reused 4 reused setup= 0.0 ms TTFB= 95.0 ms setup+TTFB= 95.0 ms websocket reused 5 reused setup= 0.0 ms TTFB= 105.0 ms setup+TTFB= 105.0 ms websocket/reused/reused: n=4 failed=0 skipped=0 min=85.0 p50=90.0 p90=105.0 p99=105.0 max=105.0 ``` All timings are in milliseconds. | Measurement | What you are timing | | - | - | | `TTFB` / `ttfb_ms` | From submitting text to receiving the first nonempty audio data, after connection setup. Includes request transmission, server processing and queuing, and the return path. | | `setup` / `setup_ms` | Hostname resolution through completed connection setup. Zero means the request reused its existing connection. | | `setup+TTFB` / `setup_plus_ttfb_ms` | The sum of connection setup and first-audio timing. Use this when assessing a request that needs a new connection. | | `dns_ms` | Elapsed time for the local `getaddrinfo()` hostname lookup, including resolver work and waiting. It does not isolate a single DNS packet's round trip. | | `tcp_connect_ms` | TCP connection time after resolution. It approximates network RTT only when one address attempt succeeds; `attempts` reports that count. | | `tls_ms` | TLS handshake time for HTTP streaming. | | `tls_and_ws_upgrade_ms` | Combined TLS handshake and WebSocket upgrade time. | | `ping_rtt_ms` | WebSocket ping/pong time, measured after the connection's last synthesis request. | TTFB stops on audio, not HTTP headers or a WebSocket `audio.start` event. Both transports return raw signed 16-bit mono PCM. The tool reads the sample rate from the response metadata. `setup+TTFB` excludes local work between connection setup and text submission. It does not measure playback startup, the time to receive all audio, or an entire voice conversation. For WebSocket, TTFB includes sending both `input.text` and `input.flush`; it does not simulate tokens arriving from an LLM. ### Interpret connection timings A first DNS lookup can take longer than later lookups. A cache miss is one possible cause, but the timer includes the whole local lookup call. You cannot attribute that delay to a particular DNS server from this measurement alone. TCP connect time and WebSocket ping RTT are different measurements. Neither isolates pure network transit, so the tool does not subtract either from TTFB to claim model inference latency. Reused requests refer to the original connection's setup and network measurements. A WebSocket handshake can finish before application authentication and session initialization. First-request TTFB includes any remaining work. Reusing a WebSocket session therefore measures a different lifecycle from sending another HTTP request on a keep-alive connection. ### Interpret percentiles and failures Percentiles use the nearest-rank method over successful generations. The tool does not discard outliers. `n` counts successes; `failed` and `skipped` show the other outcomes. > [!NOTE] > > Five requests per group provide a quick check. With fewer than 100 successful samples, p99 is the observed maximum. In the default run, both p90 and p99 equal the maximum for each group. Increase `--runs` when you need more samples, and report the sample count with your results. If a reused connection or generation fails, the tool skips the rest of that group. It does not reconnect and label later requests as warm. A failure reported after the first audio still excludes that generation from the successful TTFB summary. HTTP raw PCM has no application-level completion marker, so a normally terminated response cannot independently prove that synthesis was complete. The process exits with a nonzero status if any generation fails or is skipped. A failed WebSocket ping is reported separately from synthesis failures. See [Errors and status codes](/guides/resources/errors) when troubleshooting API failures. ## Configuration reference | Option | Default | Description | | - | - | - | | `--transport` | `both` | `http`, `websocket`, or `both`. | | `--connection` | `both` | `fresh`, `reused`, or `both`. | | `--runs` | `5` | Requests per transport and connection mode, including the first request in a reused group. | | `--model` | `CHARIOT_MODEL` or `chariot-multilingual-v1` | Model sent as the public `model_type` field. | | `--voice-id` | `CHARIOT_VOICE_ID` | Voice UUID compatible with the selected model. | | `--text` | A short English sentence | Text for each request. Surrounding whitespace is trimmed. | | `--speed` | `1.0` | A value supported by the voice's selected model in `models[].speeds`. | | `--optimize-streaming-latency` | `0` | Quality and latency setting from `0` to `4`. See [Models](/guides/models). | | `--timeout` | `60` | Seconds per TCP attempt or HTTP socket operation; WebSocket open or generation deadline. Not an overall script deadline. | | `--json` | No file | Path for raw measurements, configuration, and summaries. | | `--save-audio` | No files | Directory for a WAV file per successful request. | The tool reads `CHARIOT_API_KEY` from `.env` or your environment and authenticates with the `chariotai-api-key` header. It connects directly to `api.chariot.in` with certificate verification. It does not use environment proxy settings. ## Save and compare runs Use a different `--json` path for each run. The report includes the UTC start time, platform, Python and WebSocket library versions, model, voice, text, settings, raw trials, connection IDs, and summaries. It does not include your API key. Keep the voice, model, text, speed, and latency setting consistent when comparing results. Record your client location and network conditions alongside the JSON report. Run from the host where your application will make requests if you want to measure that path. The default order is HTTP fresh, HTTP reused, WebSocket fresh, then WebSocket reused. These groups run sequentially, so server load and network conditions can change between them. For the complete implementation and local protocol checks, see the [latency tool source and README](https://github.com/Chariot-AI/chariot-integrations/tree/main/tools/latency). This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.