Latency testing

Measure how quickly you receive the first audio from Chariot using the Python latency tool. You can test HTTP streaming and WebSocket, compare fresh and reused connections, and save the results for inspection.

Get the latency tool

Read the source, install the dependencies, and run the same measurements from your laptop or application host.

Set up

You need Git, Python 3.10 or later, a Chariot API key, and a voice that supports your selected model.

Install the tool

git clone https://github.com/Chariot-AI/chariot-integrations.git
cd chariot-integrations/tools/latency
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
cp .env.sample .env

Run these commands in a macOS or Linux shell. Keep your terminal in tools/latency for the commands below. If you already have the repository, update your checkout from main first.

Configure your key, voice, and model

Edit .env:

CHARIOT_API_KEY=your-api-key
CHARIOT_VOICE_ID=your-voice-id
CHARIOT_MODEL=chariot-multilingual-v1

Create your API key in the Dashboard. Get a voice UUID from GET /v1/voices. Choose a voice whose models[].model includes chariot-multilingual-v1.The tool defaults to Chariot Multilingual v1 and sends model_type explicitly. You can override CHARIOT_MODEL with --model. See Models for compatible voices, input limits, and speaking rates.

Run the latency test

python latency.py --model chariot-multilingual-v1 --json results.json

You receive per-request timings, connection measurements, and a TTFB summary with min, p50, p90, p99, and max. The JSON file preserves the individual measurements and configuration.

The default run makes 20 sequential synthesis requests. Normal API credits apply. There are no hidden warm-up requests or automatic retries.

Choose what to test

Both streaming endpoints are included by default:

CLI transport	Endpoint	Default requests
`http`	`POST /v1/tts/stream`	5 fresh, then 5 on a shared connection
`websocket`	`wss://api.chariot.in/v1/tts/ws`	5 fresh, then 5 on a shared session

The tool sends the same complete text for every request. It measures latency rather than concurrency, throughput, real-time factor, or playback underruns.

python latency.py --transport http --connection both --runs 5 \
  --json http-results.json

python latency.py --transport websocket --connection both --runs 5 \
  --json websocket-results.json

python latency.py --transport websocket --connection reused --runs 5 \
  --json reused-results.json

python latency.py --text "नमस्ते, आपका दिन कैसा चल रहा है?" \
  --model chariot-multilingual-v1 --save-audio audio --json results.json

--save-audio writes one WAV file per successful request after consuming the response. File writes do not enter the first-audio timing. Reusing an output path overwrites its previous results.

Fresh and reused connections

Fresh: each request opens a new connection. DNS caches and the model may already be warm.
Reused HTTP: the tool reads the complete response before sending the next request on the same HTTP/1.1 connection.
Reused WebSocket: the tool sends input.text followed by input.flush, reads through audio.done, and repeats on the same session. It sends input.done when finished.

With --runs 5, each reused group contains one request labeled first and four labeled reused. The summary keeps them separate. With --runs 1, you get no reused samples.

Read the results

The tool prints endpoint URLs before the trials. In the output, http means HTTP streaming. This illustrative excerpt shows the output format, not expected latency:

http      fresh    1 fresh  setup=   30.0 ms  TTFB=  150.0 ms  setup+TTFB=  180.0 ms
websocket reused   1 first  setup=  100.0 ms  TTFB=  180.0 ms  setup+TTFB=  280.0 ms
websocket reused   2 reused setup=    0.0 ms  TTFB=   85.0 ms  setup+TTFB=   85.0 ms
websocket reused   3 reused setup=    0.0 ms  TTFB=   90.0 ms  setup+TTFB=   90.0 ms
websocket reused   4 reused setup=    0.0 ms  TTFB=   95.0 ms  setup+TTFB=   95.0 ms
websocket reused   5 reused setup=    0.0 ms  TTFB=  105.0 ms  setup+TTFB=  105.0 ms

websocket/reused/reused: n=4 failed=0 skipped=0 min=85.0 p50=90.0 p90=105.0 p99=105.0 max=105.0

All timings are in milliseconds.

Measurement	What you are timing
`TTFB` / `ttfb_ms`	From submitting text to receiving the first nonempty audio data, after connection setup. Includes request transmission, server processing and queuing, and the return path.
`setup` / `setup_ms`	Hostname resolution through completed connection setup. Zero means the request reused its existing connection.
`setup+TTFB` / `setup_plus_ttfb_ms`	The sum of connection setup and first-audio timing. Use this when assessing a request that needs a new connection.
`dns_ms`	Elapsed time for the local `getaddrinfo()` hostname lookup, including resolver work and waiting. It does not isolate a single DNS packet’s round trip.
`tcp_connect_ms`	TCP connection time after resolution. It approximates network RTT only when one address attempt succeeds; `attempts` reports that count.
`tls_ms`	TLS handshake time for HTTP streaming.
`tls_and_ws_upgrade_ms`	Combined TLS handshake and WebSocket upgrade time.
`ping_rtt_ms`	WebSocket ping/pong time, measured after the connection’s last synthesis request.

TTFB stops on audio, not HTTP headers or a WebSocket audio.start event. Both transports return raw signed 16-bit mono PCM. The tool reads the sample rate from the response metadata. setup+TTFB excludes local work between connection setup and text submission. It does not measure playback startup, the time to receive all audio, or an entire voice conversation. For WebSocket, TTFB includes sending both input.text and input.flush; it does not simulate tokens arriving from an LLM.

Interpret connection timings

A first DNS lookup can take longer than later lookups. A cache miss is one possible cause, but the timer includes the whole local lookup call. You cannot attribute that delay to a particular DNS server from this measurement alone. TCP connect time and WebSocket ping RTT are different measurements. Neither isolates pure network transit, so the tool does not subtract either from TTFB to claim model inference latency. Reused requests refer to the original connection’s setup and network measurements. A WebSocket handshake can finish before application authentication and session initialization. First-request TTFB includes any remaining work. Reusing a WebSocket session therefore measures a different lifecycle from sending another HTTP request on a keep-alive connection.

Interpret percentiles and failures

Percentiles use the nearest-rank method over successful generations. The tool does not discard outliers. n counts successes; failed and skipped show the other outcomes.

Five requests per group provide a quick check. With fewer than 100 successful samples, p99 is the observed maximum. In the default run, both p90 and p99 equal the maximum for each group. Increase --runs when you need more samples, and report the sample count with your results.

If a reused connection or generation fails, the tool skips the rest of that group. It does not reconnect and label later requests as warm. A failure reported after the first audio still excludes that generation from the successful TTFB summary. HTTP raw PCM has no application-level completion marker, so a normally terminated response cannot independently prove that synthesis was complete. The process exits with a nonzero status if any generation fails or is skipped. A failed WebSocket ping is reported separately from synthesis failures. See Errors and status codes when troubleshooting API failures.

Configuration reference

Option	Default	Description
`--transport`	`both`	`http`, `websocket`, or `both`.
`--connection`	`both`	`fresh`, `reused`, or `both`.
`--runs`	`5`	Requests per transport and connection mode, including the first request in a reused group.
`--model`	`CHARIOT_MODEL` or `chariot-multilingual-v1`	Model sent as the public `model_type` field.
`--voice-id`	`CHARIOT_VOICE_ID`	Voice UUID compatible with the selected model.
`--text`	A short English sentence	Text for each request. Surrounding whitespace is trimmed.
`--speed`	`1.0`	A value supported by the voice’s selected model in `models[].speeds`.
`--optimize-streaming-latency`	`0`	Quality and latency setting from `0` to `4`. See Models.
`--timeout`	`60`	Seconds per TCP attempt or HTTP socket operation; WebSocket open or generation deadline. Not an overall script deadline.
`--json`	No file	Path for raw measurements, configuration, and summaries.
`--save-audio`	No files	Directory for a WAV file per successful request.

The tool reads CHARIOT_API_KEY from .env or your environment and authenticates with the chariotai-api-key header. It connects directly to api.chariot.in with certificate verification. It does not use environment proxy settings.

Save and compare runs

Use a different --json path for each run. The report includes the UTC start time, platform, Python and WebSocket library versions, model, voice, text, settings, raw trials, connection IDs, and summaries. It does not include your API key. Keep the voice, model, text, speed, and latency setting consistent when comparing results. Record your client location and network conditions alongside the JSON report. Run from the host where your application will make requests if you want to measure that path. The default order is HTTP fresh, HTTP reused, WebSocket fresh, then WebSocket reused. These groups run sequentially, so server load and network conditions can change between them. For the complete implementation and local protocol checks, see the latency tool source and README.

Get started

Text to speech

Speech to text

Resources

Get the latency tool

Set up

Choose what to test

Fresh and reused connections

Read the results

Interpret connection timings

Interpret percentiles and failures

Configuration reference

Save and compare runs

Get the latency tool

​Set up

​Choose what to test

​Fresh and reused connections

​Read the results

​Interpret connection timings

​Interpret percentiles and failures

​Configuration reference

​Save and compare runs

Set up

Choose what to test

Fresh and reused connections

Read the results

Interpret connection timings

Interpret percentiles and failures

Configuration reference

Save and compare runs