Idempotency - Chattler

Both chat endpoints, POST /v1/agents/{agent_id}/chat and POST /v1/agents/{agent_id}/chat/stream, require an Idempotency-Key header. A request without one is 422 validation_error.

Choosing a key

Any value of 1-255 characters that is unique per logical request. A UUID generated at the moment you decide to send a message is the simplest choice. Keys are scoped to your API key, so you do not need to worry about collisions with other keys.

Idempotency-Key: 7f20a5f0-5dde-48cf-94aa-38acbbd218e0

What a retry returns

Retrying with the same Idempotency-Key and the same body returns the stored result: the original request_id, the same assistant message and the same usage. No second reply is generated and nothing is charged again. Records live for 24 hours. After that the same key starts a fresh request.

The stored result is what the runtime persisted anyway: the assistant message tagged with the request id and the LLM usage billed under that id. A replay therefore never calls the model or touches the balance, even if the first attempt died halfway through.

Conflicts

Status	Code	When
`409`	`idempotency_in_progress`	The first attempt is still running. Wait for `Retry-After` seconds (2 by default) and retry with the same key.
`409`	`idempotency_conflict`	The same key was sent with a different body (a different `external_user_id` or `message`). Use a new key for the new request.

Streaming and disconnects

The rules are the same on the streaming endpoint. If your client disconnects mid-stream, the reply keeps being produced and stored. A retry with the same key streams response.created followed directly by response.completed carrying the stored result, without replaying the deltas. See Streaming.

Recommended retry loop

import time
import uuid

import httpx


def chat_with_retry(external_user_id: str, message: str, attempts: int = 5) -> dict:
    idempotency_key = str(uuid.uuid4())  # one key for the whole logical request
    for _ in range(attempts):
        response = httpx.post(
            f"{BASE_URL}/v1/agents/{AGENT_ID}/chat",
            headers={**HEADERS, "Idempotency-Key": idempotency_key},
            json={"external_user_id": external_user_id, "message": message},
            timeout=120,
        )
        if response.status_code < 400:
            return response.json()
        code = response.json()["error"]["code"]
        if code in ("idempotency_in_progress", "conversation_busy", "rate_limit_exceeded",
                    "api_rate_limit_unavailable", "service_unavailable"):
            time.sleep(float(response.headers.get("Retry-After", "2")))
            continue
        raise RuntimeError(code)
    raise TimeoutError("gave up")

const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
const RETRYABLE = new Set([
  "idempotency_in_progress",
  "conversation_busy",
  "rate_limit_exceeded",
  "api_rate_limit_unavailable",
  "service_unavailable",
]);

async function chatWithRetry(externalUserId, message, attempts = 5) {
  const idempotencyKey = crypto.randomUUID(); // one key for the whole logical request
  for (let i = 0; i < attempts; i++) {
    const response = await fetch(`${baseUrl}/v1/agents/${agentId}/chat`, {
      method: "POST",
      headers: {
        Authorization: `Bearer ${apiKey}`,
        "Idempotency-Key": idempotencyKey,
        "Content-Type": "application/json",
      },
      body: JSON.stringify({ external_user_id: externalUserId, message }),
    });
    const body = await response.json();
    if (response.ok) return body;
    if (!RETRYABLE.has(body.error.code)) throw new Error(body.error.code);
    await sleep(Number(response.headers.get("Retry-After") ?? 2) * 1000);
  }
  throw new Error("gave up");
}