Retries and idempotency

A call can fail, or its answer can be lost on the way back. Send each call with an Idempotency-Key, and you can send it again safely. A decision that already ran returns its first answer, and isn’t charged again. Zero data retention is the exception.

export SQWISH_OPERATION_KEY='refund-check-4471'

curl --fail-with-body --connect-timeout 5 --max-time 40 \
  https://api.sqwish.ai/v1/decide \
  -H "Authorization: Bearer $SQWISH_API_KEY" \
  -H "Idempotency-Key: $SQWISH_OPERATION_KEY" \
  -H 'Content-Type: application/json' \
  --data-binary '{"model": "kettei-core", "context": "Please refund the duplicate charge on order 4471.", "decisions": [{"id": "refund", "kind": "binary", "question": "Is a refund requested?"}]}'

If the answer doesn’t arrive, run the same curl command again. Keep the key and the body exactly as they were.

When to retry

Retry when error.retryable is true. Also retry when the reply never arrived and you still have the Idempotency-Key. Check error.code as well before you repeat a call (Error codes).

If the response has a Retry-After header, wait that long.
Otherwise wait 0.25 * 2^attempt seconds. Add a little random jitter, so that many clients don’t retry at the same moment.
Cap the wait.
Give the call a budget of attempts or elapsed time. When the budget runs out, stop. Keep the operation’s record, so that you can recover it later.

Don’t retry a 402, a 422 or any other error with retryable: false until you have changed the account or the request. A corrected request is a new operation, so give it a new key.

Case	One possible run
A brief failure	A `503` with `retryable: true` and `Retry-After: 5` at 0 s, and again at 5 s. Then a `200` at 10 s. All three attempts use the same key and body, and the success is billed once
Rate limited	A `429` with `Retry-After: 2` at 0 s, then a `200` at 2 s. `Retry-After` wins over the exponential wait. The limit itself isn’t a charge
Not retryable	A `503` with `retryable: false`, such as a model this server doesn’t serve. Stop backing off. Change the model or fix the request, then send it with a new `Idempotency-Key`

This sample follows the same policy for a plain httpx call, with a cap of 8 seconds:

import os
import random
import time
import httpx

def decide(body, operation_key, attempts=5):
    headers = {
        "Authorization": f"Bearer {os.environ['SQWISH_API_KEY']}",
        "Idempotency-Key": operation_key,
    }
    for attempt in range(attempts):
        try:
            response = httpx.post(
                "https://api.sqwish.ai/v1/decide",
                headers=headers,
                json=body,
                timeout=40.0,
            )
        except httpx.TransportError:
            if attempt == attempts - 1:
                raise
            time.sleep(min(8, 0.25 * 2**attempt) + random.random() * 0.25)
            continue
        if response.status_code == 200:
            return response.json()
        try:
            retryable = response.json()["error"]["retryable"]
        except (ValueError, KeyError, TypeError):
            # A proxy in front of the API can fail without the error envelope.
            retryable = response.status_code in (502, 503, 504)
        if not retryable or attempt == attempts - 1:
            response.raise_for_status()
        retry_after = response.headers.get("Retry-After")
        wait = float(retry_after) if retry_after else min(8, 0.25 * 2**attempt)
        time.sleep(wait + random.random() * 0.25)

If an operation ended in failure, its key returns that failure again. The operation doesn’t run a second time.

Idempotency keys

Make one Idempotency-Key for each operation you intend, and save it with the request. Retry that operation with the same key and unchanged input. For a raw upload, keep its exact bytes and name. The API reference shows the header only on the operations that support it.

	Creating resources	Inference
Operations	Datasets in JSON or JSONL, rows added to a dataset, fine-tuning jobs, continuing a family, labelling sources and jobs, labelling finalisation and prompt-tuning jobs	`/v1/decide`, `/v1/systemone` and the Claude Code hook
A replay returns	The existing resource. A job comes back in its current state, which may no longer be queued	The original outcome, without running or charging again. The models that answer, the fallback choices, the tokenizer and the prices are the ones the first request was accepted with
`Idempotency-Replayed: true`	On a replayed fine-tuning job or family continuation, never on the first request	On every replay
Changed input, same key	A conflict	A conflict

For inference:

A successful answer to a request sent with a key is saved, with its charge, before it is returned. It is kept for 24 hours.
A request sent without a key keeps no answer.
An organisation with zero data retention keeps no answer either (Zero data retention).
An operation that is still running returns request_in_progress, with retry guidance.
A replay of a failed operation returns that failure. It is not a new attempt.
After 24 hours the saved answer expires, but the operation remains. Its key then returns idempotency_response_expired, and never runs the decision again.

After a lost response

Send the same request again, with the same key and the same body.

Decisions

For POST /v1/decide, /v1/systemone and the Claude Code hook, save a unique Idempotency-Key with the exact request before the first attempt. If the reply is lost, resend both unchanged. A replay uses the models and prices the first request was accepted with. This holds even if its family moved to a newer model in the meantime. A replay isn’t charged again. Idempotency keys says what a replay returns while the call runs, after it fails and after 24 hours. Reuse a key only to recover a call. Use a new key when you want the decision to run again. Each replay attempt uses the account’s read budget (Rate limits). That includes pending, conflicting, failed and expired attempts. Respect Retry-After, and back off within a budget. Tie downstream actions, such as assigning a ticket, to your application’s own operation ID. Then a retry can’t repeat them. Your request history helps you diagnose an operation, but it doesn’t hold the saved answer. While the answer is kept, the original request with its key returns it. Keep the request IDs and the charge details.

Datasets and jobs

A create that supports idempotency recovers the same way. A later attempt with the same key settles an uncertain outcome. This example creates a dataset. Save it as dataset.json:

{
  "name": "Retry format example",
  "examples": [
    {"context":"Please refund the duplicate charge on order 4471.","decisions":[{"id":"refund","kind":"binary","question":"Is a refund requested?"}],"targets":{"refund":{"yes":1,"no":0}}},
    {"context":"Where is the parcel for order 2280?","decisions":[{"id":"refund","kind":"binary","question":"Is a refund requested?"}],"targets":{"refund":{"yes":0,"no":1}}}
  ]
}

Choose one key for this dataset. Save the key and the body before you send them. Keep both unchanged across retries and process restarts.

export SQWISH_OPERATION_KEY='retry-format-example-v1'

curl --fail-with-body --connect-timeout 5 --max-time 40 \
  https://api.sqwish.ai/v1/datasets \
  -H "Authorization: Bearer $SQWISH_API_KEY" \
  -H "Idempotency-Key: $SQWISH_OPERATION_KEY" \
  -H 'Content-Type: application/json' --data-binary @dataset.json

If the reply is lost, run the same request again. It returns the existing dataset instead of creating another one. Changed input with the same key gets an idempotency conflict. To create another dataset on purpose, use a new key.

Zero data retention

An organisation with zero data retention keeps no answer. So a key isn’t honoured there. A retry runs the request again, and is charged again. That includes a retry an SDK makes by itself after a network error.

Switches, keep warm and API keys

Switches and keep warm have no general idempotency-key contract. After a lost response, read the family or model before you repeat the call.

A switch that is already where you asked changes nothing.
served or adapter refuse the call if the model moved since you looked (Switch a family or a model on).

After a lost response to making an API key, you can’t read the secret from the key list. Check the keys, and revoke any you don’t want, before you make another on purpose (API keys).

Polling timeouts

A timeout while you poll a job only stops your client waiting. The job can still be running. Read it again, or cancel it explicitly if that is what you want.

With the SDKs

The Python and TypeScript clients, sqwishai on PyPI and npm, follow this policy for eligible calls. They wait as Retry-After says, or back off for about 4 seconds at most, and stop after two retries. Retries explains their idempotency keys and options.

Get started

Make decisions

Improve accuracy

Use the API

Account and billing

Privacy and security

Integrations

Retries and idempotency

When to retry

Idempotency keys

After a lost response

Decisions

Datasets and jobs

Zero data retention

Switches, keep warm and API keys

Polling timeouts

With the SDKs

​When to retry

​Idempotency keys

​After a lost response

​Decisions

​Datasets and jobs

​Zero data retention

​Switches, keep warm and API keys

​Polling timeouts

​With the SDKs

When to retry

Idempotency keys

After a lost response

Decisions

Datasets and jobs

Zero data retention

Switches, keep warm and API keys

Polling timeouts

With the SDKs