> ## Documentation Index
> Fetch the complete documentation index at: https://docs.sqwish.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Improve a decision

> Label examples, measure the base model, tune the wording and train a model of your own.

A base model does well when the question is clear. To do better on your own traffic, give it
examples whose answers you know. This page takes the [quickstart](/quickstart)'s team question
through each step with the Python client. The TypeScript client has the same calls in camelCase
([SDKs](/guides/sdks)), and each step's guide shows the HTTP request.

## 1. Build a dataset

Each row is a decide request, its `context` and `decisions`, plus `targets`: the answer you
expect, as a probability for every outcome. Write one row per line of a JSONL file:

```json tickets.jsonl theme={null}
{"context": "Please refund the duplicate charge on order 4471.", "decisions": [{"id": "team", "kind": "single", "question": "Which team should handle this ticket?", "outcomes": {"billing": "Charges, invoices, and financial exports.", "technical": "Other application failures.", "sales": "New purchases."}}], "targets": {"team": {"billing": 1, "technical": 0, "sales": 0}}}
```

Word each decision exactly as your application asks it. Some contexts are held out to measure
models, and never train them ([the split](/guides/datasets#choose-the-evaluation-set)).

```python theme={null}
from sqwishai import OneHot

# The decision every row asks, worded as in the file; the steps below use it again.
team = {"id": "team", "kind": "single", "question": "Which team should handle this ticket?",
        "outcomes": {"billing": "Charges, invoices, and financial exports.",
                     "technical": "Other application failures.",
                     "sales": "New purchases."}}

client = OneHot()
dataset = client.datasets.upload("tickets.jsonl", name="Support tickets",
                                 idempotency_key="support-tickets-v1")
```

No labels yet? Teacher models can label real queries for you to review
([review teacher labels](/guides/labelling)). [Build a dataset](/guides/datasets) covers the
format, rewards and the split.

## 2. Measure the base model

```python theme={null}
import time

evaluation = client.evaluations.create(dataset["id"], models=["kettei-core"])["data"][0]
while evaluation["status"] in ("queued", "running"):
    time.sleep(5)
    evaluation = client.evaluations.get(evaluation["id"])
print(evaluation["metrics"])
```

`metrics` has accuracy, log loss and calibration error on the held-out rows. An evaluation is
charged as decisions ([compare models](/guides/datasets#compare-models-on-your-data)).

## 3. Try clearer wording

Prompt tuning rewrites the question and the outcome descriptions, then tests the rewrite once on
rows it never saw. It needs enough labelled rows for each outcome
([limits](/api#requests-and-limits)).

```python theme={null}
job = client.prompt_tuning.create(dataset["id"], decision="team",
                                  idempotency_key="support-team-wording-v1")
job = client.prompt_tuning.wait(job["id"])
if job["result"]["verdict"] == "improved":
    team = job["result"]["tuned"]  # use this wording in new requests
```

Use the new wording only when the verdict is `improved` ([tune the wording](/guides/prompt-tuning)).

## 4. Train a model of your own

A fine-tune trains a model on your rows and starts a family, a named line of models such as
`support-router`. The new model joins only if it passes its gate, a check for regressions against
the model it started from on the held-out rows. A family needs enough training rows, and enough
held-out decisions for the gate to judge ([what the start checks](/guides/fine-tuning#what-the-start-checks)).

```python theme={null}
quote = client.fine_tuning.estimate(dataset["id"], family="support-router")
job = client.fine_tuning.create(dataset["id"], family="support-router",
                                max_price_usd=quote["price_usd"],
                                idempotency_key="support-router-v1")
job = client.fine_tuning.wait(job["id"], timeout=6 * 3600)

if job["outcome"] == "joined":
    client.families.wait("support-router", job["output_model"])
    result = client.decide(context="Where is my refund for order 4471?", decisions=[team],
                           model="support-router")
```

The estimate is the exact fee, and a fine-tune is charged only if it succeeds
([pricing](/guides/pricing)). Calls to `support-router` get the family's latest model that
passed. [Fine-tune a family](/guides/fine-tuning) shows how to read the gate.

## 5. Keep improving

Add newly labelled rows to the dataset as its next version, then continue the family. The next
model trains from the latest one and joins only if it passes its gate too.

```python theme={null}
client.datasets.upload_rows(dataset["id"], "more-tickets.jsonl", family="support-router",
                            idempotency_key="support-tickets-v2")
quote = client.fine_tuning.estimate_continue("support-router")
job = client.fine_tuning.continue_family("support-router",
                                         dataset_version=quote["dataset_version"],
                                         max_price_usd=quote["price_usd"],
                                         idempotency_key="support-router-v2")
```

To go back, roll the family back to an earlier model, which discards the newer ones:
`client.families.rollback("support-router", "support-router@1", ["support-router@2"])`.
[Families, models and datasets](/guides/families) covers switching, keeping a model warm and
deleting.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.