Rate limits & concurrency

​

How concurrency works

• Text to speech: the limit applies to both HTTP TTS endpoints and active WebSocket generations. Idle WebSocket connections do not hold a generation slot. WebSocket limit errors close the connection with 4429.
• Speech to text: live, synchronous file, and asynchronous file transcription share the same ASR limit. Each streaming session holds one slot for its entire life, including pauses, until the connection closes. A new session over the limit gets an error event and closes with 4429. Synchronous file requests return 429 when all slots are busy. Async batch acceptance does not reserve slots; each file takes a slot when a worker starts preparing and transcribing it.

​

Limits by plan

​

The 429 response

{
  "statusCode": 429,
  "error": "HTTPException",
  "message": "Concurrent request limit exceeded (2) for tts.standard on the current plan"
}

{
  "statusCode": 403,
  "error": "HTTPException",
  "message": "Product model tts.standard is not available on the current plan"
}

​

Handling limits

• Cap your in-flight requests to your plan’s concurrency limit so you rarely hit 429 in the first place.
• Retry with exponential backoff and jitter when you do, e.g. 0.5s, 1s, 2s, 4s with random jitter.
• Use a worker pool sized to your limit rather than firing all requests at once.
• Close speech-to-text sessions promptly. Send input.done when the audio ends, rather than leaving an idle session open until it times out.

import time, random, requests

def synthesize_with_retry(payload, key, max_retries=5):
    for attempt in range(max_retries):
        r = requests.post(
            "https://api.chariot.in/v1/tts",
            headers={"chariotai-api-key": key},
            json=payload,
        )
        if r.status_code != 429:
            r.raise_for_status()
            return r
        sleep = (2 ** attempt) * 0.5 + random.random() * 0.25
        time.sleep(sleep)
    raise RuntimeError("Concurrency limit exceeded after retries")

​

Need more headroom?