> ## Documentation Index
> Fetch the complete documentation index at: https://docs.sqwish.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Score many texts

> Score thousands of comments, tickets or documents within your rate limits, price them first and rerun safely.

There is no batch endpoint. Send one request for each text, and ask every question about that text in
the same request. Then send the requests at a steady pace that stays within your tier's limits.

Keep each text in a request of its own. Decisions in one request can affect each other, so texts packed
into one context sway each other's answers, and accuracy drops. To go faster, move up a tier instead
([move up a tier](https://docs.sqwish.ai/guides/limits.md#move-up-a-tier)).

## Price it first

Ask for an estimate with a typical text and your decisions. It counts the input tokens and quotes the
most the call can cost. It runs no model and charges nothing.

```python Python theme={null}
from decimal import Decimal
from sqwishai import OneHot

decisions = [{"id": "spam", "kind": "binary", "question": "Is the comment spam?"}]

with OneHot() as client:
    quote = client.estimate(context="Where can I buy this jacket?", decisions=decisions)

print(quote["input_tokens"], quote["maximum_charge_usd"])
print("100,000 comments:", Decimal(quote["maximum_charge_usd"]) * 100_000)
```

```typescript TypeScript theme={null}
import { OneHot, type Decision } from "sqwishai";

const decisions: Decision[] = [{ id: "spam", kind: "binary", question: "Is the comment spam?" }];
const client = new OneHot();
const quote = await client.estimate({ context: "Where can I buy this jacket?", decisions });

console.log(quote.input_tokens, quote.maximum_charge_usd);
```

The estimate is `POST /v1/decide/estimate`, with the same body as a decision. Estimates count against
their own token budget, `estimate_tokens_per_minute`. [Pricing](https://docs.sqwish.ai/guides/pricing.md) has the price for each
model.

## Know your limits

Three limits of your tier set how fast you can go. They are shared by every key of your organisation.

| Limit | What it counts |
| - | - |
| Requests a minute, `rpm` | Every decision request you send |
| Input tokens a minute, `tpm` | The billed tokens of those requests |
| Operations at once, `concurrency` | Requests still running |

Short texts usually meet the requests limit first, and long texts the tokens limit. With one text a request,
your requests a minute set your pace: multiply by 60 for texts an hour. At tier 0 that is 1,800 an hour,
so 100,000 texts take more than two days.

A new organisation starts at tier 0. [Limits](https://docs.sqwish.ai/guides/limits.md#tiers) has the numbers for each tier, and buying
credit moves you to a higher one ([move up a tier](https://docs.sqwish.ai/guides/limits.md#move-up-a-tier)).

`GET /v1/account` gives your own limits in `effective_limits`. It needs a key with full or read access:
an inference key can't read it.

## Send them at a steady pace

This code runs as many workers as your tier allows at once. Each worker waits between its calls, so
together they stay under your requests limit. It uses [`AsyncOneHot`](https://docs.sqwish.ai/guides/sdks#async-python), the
async Python client.

```python Python theme={null}
import asyncio
import time
from sqwishai import AsyncOneHot

RPM = 30           # your tier's requests a minute: 30 at tier 0
CONCURRENCY = 1    # your tier's operations at once: 1 at tier 0

async def score_all(items, decisions):
    results = [None] * len(items)
    todo = iter(range(len(items)))
    async with AsyncOneHot() as client:
        async def worker():
            for i in todo:
                started = time.monotonic()
                try:
                    results[i] = await client.decide(
                        context=items[i]["text"], decisions=decisions,
                        idempotency_key=f"comment-{items[i]['id']}-v1")
                except Exception as error:  # keep it, and send it again later
                    results[i] = error
                await asyncio.sleep(max(0, 60 * CONCURRENCY / RPM - (time.monotonic() - started)))
        await asyncio.gather(*(worker() for _ in range(CONCURRENCY)))
    return results
```

```typescript TypeScript theme={null}
import { OneHot, type Decision } from "sqwishai";

const RPM = 30;         // your tier's requests a minute: 30 at tier 0
const CONCURRENCY = 1;  // your tier's operations at once: 1 at tier 0
const client = new OneHot();
const sleep = (ms: number) => new Promise((resolve) => setTimeout(resolve, ms));

async function scoreAll(items: { id: string; text: string }[], decisions: Decision[]) {
  const results: unknown[] = new Array(items.length);
  let next = 0;
  async function worker() {
    for (let i = next++; i < items.length; i = next++) {
      const started = Date.now();
      results[i] = await client
        .decide({ context: items[i].text, decisions },
                { idempotencyKey: `comment-${items[i].id}-v1` })
        .catch((error) => error); // keep it, and send it again later
      await sleep(Math.max(0, (60_000 * CONCURRENCY) / RPM - (Date.now() - started)));
    }
  }
  await Promise.all(Array.from({ length: CONCURRENCY }, worker));
  return results;
}
```

Set `RPM` and `CONCURRENCY` to your tier's numbers. For long texts or images, input tokens a minute
(`tpm`) can run out first: lower `RPM` until `RPM` times a request's input tokens stays under
`tpm`. Over a limit, a request gets `429` with a
`Retry-After` header. The clients wait that long and try again, twice by default
([retries](https://docs.sqwish.ai/guides/sdks.md#retries)).

For a long run, write each answer to a file as it arrives, with its text's ID. When you start again,
skip the IDs you already have.

## Rerun safely

Give each text its own [idempotency key](https://docs.sqwish.ai/guides/retries.md#idempotency-keys), made from its ID, as above.

* Run the same items again, and each one already answered returns its saved answer. It isn't run or
  charged again. A saved answer lasts 24 hours. After that, its key answers
  `409 idempotency_response_expired` and never runs the text again.
* Change the decisions, and change the key too, such as `-v2`. The same key with a different body is
  refused with `409 idempotency_conflict`.
* An organisation with [zero data retention](https://docs.sqwish.ai/guides/zero-data-retention.md) keeps no answers, so a rerun
  there runs and charges again.

The clients have already retried an error that can succeed later. If an item still failed, read its
error. Send it again with the same key when `retryable` is true. Otherwise the same key returns the same
failure: fix the request, and give it a new key ([errors](https://docs.sqwish.ai/api.md#error-codes)).

## Keep each request lean

* Put only the text the questions need in `context`. You pay for every input token.
* Ask all the questions about a text in one request. You pay for the text once.
* `kettei-core` is the cheapest model, and it reads text. Use `kettei-chub` or `kettei-fat` for images.
* Keep the questions and outcome descriptions the same for every text, so the answers are comparable.

## Before a long run

* **Have enough credit.** Multiply the estimate by the number of texts. When credit runs out, the rest
  get `402 insufficient_credit`, and nothing is charged for them. Buy credit, or turn on automatic
  top-ups ([buy credit](https://docs.sqwish.ai/guides/billing.md#buy-credit)), then send them again.
* **Warm a fine-tuned model first.** A fine-tuned model that has been idle may need to load. With the
  default fallback, a call that waits more than 10 seconds for the load is answered by its base model,
  with `fallback.reason` `model_warming`. Keep the model warm for the run
  ([keep a model warm](https://docs.sqwish.ai/guides/keep-warm.md)), or check `fallback` in each answer and send those texts
  again with a new idempotency key. The old key would return the saved fallback answer.

## Check a sample first

Before you score everything, score a few hundred texts whose answers you know, and check the answers
([compare models](https://docs.sqwish.ai/guides/evaluations.md)). [Analyse every comment](https://docs.sqwish.ai/examples/comments.md) puts all of this
together for the comments on a post.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.