Decide when to hand a conversation on

A cheap bot can finish many conversations, but some need a person or a stronger model. This example checks the bot’s next reply and says whether to send it or hand the conversation on. Use it at the end of your cheap path, before you pay for the expensive one.

How it works

The context is JSON with the conversation so far and the reply the bot wants to send next, bot_next_reply. The keys are yours to choose. One decision, route, asks whether the bot can safely finish the conversation with that reply. It has two outcomes: handle means the reply resolves the issue, and escalate means a person or a stronger model should take over. A policy turns those into one of your two actions, keep or hand_off, using what each mistake costs:

Your action	Cost if the answer is `handle`	Cost if the answer is `escalate`
`keep`: send the bot’s reply	0	10
`hand_off`: pass the conversation on	1	0

Keeping a conversation that should have gone up costs ten times as much as a needless hand-off. So policy.action is hand_off whenever the probability of escalate is above 1 in 11.

Try it

Open this recipe in the playground to run it without an account. The customer there was charged £1,200 instead of £120 and will dispute it with the bank today. The bot’s next reply says refunds take 5 to 7 working days. To run it from your code, set SQWISH_API_KEY as in the quickstart, then send this request:

from sqwishai import OneHot

with OneHot() as client:  # reads SQWISH_API_KEY
    result = client.decide(
        model="kettei-core",
        context={
            "conversation": [
                {
                    "from": "customer",
                    "text": "I was charged £1,200 instead of £120. I need this fixed today or I'll dispute it with my bank.",
                },
                {
                    "from": "bot",
                    "text": "Sorry about that. I can see a charge of £1,200 on 3 October.",
                },
            ],
            "bot_next_reply": "Refunds usually take 5 to 7 working days. Is there anything else I can help with?",
        },
        decisions=[
            {
                "id": "route",
                "kind": "single",
                "question": "Can the bot safely finish this conversation with its next reply?",
                "outcomes": {
                    "handle": "The reply resolves the issue and nothing risky is left.",
                    "escalate": "A person or a stronger model should take over.",
                },
                "policy": {
                    "action_costs": {
                        "keep": {
                            "handle": 0,
                            "escalate": 10,
                        },
                        "hand_off": {
                            "handle": 1,
                            "escalate": 0,
                        },
                    },
                },
            },
        ],
    )

import { OneHot } from "sqwishai";

const client = new OneHot(); // reads SQWISH_API_KEY
const result = await client.decide({
  "model": "kettei-core",
  "context": {
    "conversation": [
      {
        "from": "customer",
        "text": "I was charged £1,200 instead of £120. I need this fixed today or I'll dispute it with my bank."
      },
      {
        "from": "bot",
        "text": "Sorry about that. I can see a charge of £1,200 on 3 October."
      }
    ],
    "bot_next_reply": "Refunds usually take 5 to 7 working days. Is there anything else I can help with?"
  },
  "decisions": [
    {
      "id": "route",
      "kind": "single",
      "question": "Can the bot safely finish this conversation with its next reply?",
      "outcomes": {
        "handle": "The reply resolves the issue and nothing risky is left.",
        "escalate": "A person or a stronger model should take over."
      },
      "policy": {
        "action_costs": {
          "keep": {
            "handle": 0,
            "escalate": 10
          },
          "hand_off": {
            "handle": 1,
            "escalate": 0
          }
        }
      }
    }
  ]
});

curl --fail-with-body --silent --show-error https://api.sqwish.ai/v1/decide \
  -H "Authorization: Bearer $SQWISH_API_KEY" \
  -H "Content-Type: application/json" \
  --data-binary @- <<'JSON'
{
  "model": "kettei-core",
  "context": {
    "conversation": [
      {
        "from": "customer",
        "text": "I was charged £1,200 instead of £120. I need this fixed today or I'll dispute it with my bank."
      },
      {
        "from": "bot",
        "text": "Sorry about that. I can see a charge of £1,200 on 3 October."
      }
    ],
    "bot_next_reply": "Refunds usually take 5 to 7 working days. Is there anything else I can help with?"
  },
  "decisions": [
    {
      "id": "route",
      "kind": "single",
      "question": "Can the bot safely finish this conversation with its next reply?",
      "outcomes": {
        "handle": "The reply resolves the issue and nothing risky is left.",
        "escalate": "A person or a stronger model should take over."
      },
      "policy": {
        "action_costs": {
          "keep": {
            "handle": 0,
            "escalate": 10
          },
          "hand_off": {
            "handle": 1,
            "escalate": 0
          }
        }
      }
    }
  ]
}
JSON

Act on the answer

The outcomes are handle and escalate, but your code acts on policy.action, which is keep or hand_off:

action = result["decisions"]["route"]["policy"]["action"]

if action == "hand_off":
    ...  # pass the conversation to a person or a bigger model
else:
    ...  # "keep": send the bot's reply and end the turn

const action = result.decisions.route.policy?.action;

if (action === "hand_off") {
  // pass the conversation to a person or a bigger model
} else {
  // "keep": send the bot's reply and end the turn
}

policy.expected_costs shows what each action was expected to cost.

Adapt it to your data

Your costs. Set them from your own numbers: what a missed hand-off costs you, against the price of a person’s time or a bigger model’s call. Only how they compare matters: 10 and 1 act the same as 100 and 10 (weigh what mistakes cost).
Your conversations. Send your own conversation in the same shape, with the reply your bot is about to send.
Many conversations. Send one request per conversation.

What to watch

Track two numbers:

Missed hand-offs: conversations the bot kept that a person later had to take over. These cost you most. If there are too many, raise the cost of keep when the answer is escalate.
The hand-off rate: the share of conversations passed on. This is what you spend on people or the bigger model. If it is too high, lower the cost of keep when the answer is escalate, or raise the cost of hand_off when the answer is handle.

How well it works

This recipe hasn’t been measured on a public dataset. Its sample shows the format: measure it on your own examples before you rely on it.

To measure it, start with conversations a person took over: they show the hand-offs it should catch.

Next steps

Route a support ticket: send each ticket to the right team.
Improve a decision: measure it on your own conversations and make it more accurate.
API reference: every field in the request and the answer.

Overview

Triage and routing

Agents

Retrieval and answer quality

Decide when to hand a conversation on

How it works

Try it

Act on the answer

Adapt it to your data

What to watch

How well it works

Next steps