Models

Keep a model warm

Keep a model loaded between requests on its own, by its number such as router@4, or stop. Its own switch must be on: 409 model_off otherwise, and switching it off stops keep warm too. Switching on loads and probes it first, then holds its first hour, at most max_price_usd. A model kept warm through its family as well takes one warm place and is billed once. switch_off stops keep warm on families or models in the same step and leaves their switches on, so a request at the limit can swap. Past the account’s limit, 409 warm_limit lists the models still kept warm after that, least recently used first (used_at), each with the keep warms that hold it: its own (keep_warm) and its families’ (families). A size with no kept-warm place left answers 503 warm_capacity, unless switch_off stops a model of that size. With adapter, only while the number names the model the caller saw: 409 model_changed otherwise, before anything loads.

POST

models

{model}

keep-warm

Keep a model warm

curl --request POST \
  --url https://console.sqwish.ai/v1/models/{model}/keep-warm \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '
{
  "enabled": true,
  "max_price_usd": "<string>",
  "adapter": "<string>",
  "switch_off": [
    "<string>"
  ]
}
'

import requests

url = "https://console.sqwish.ai/v1/models/{model}/keep-warm"

payload = {
    "enabled": True,
    "max_price_usd": "<string>",
    "adapter": "<string>",
    "switch_off": ["<string>"]
}
headers = {
    "Authorization": "Bearer <token>",
    "Content-Type": "application/json"
}

response = requests.post(url, json=payload, headers=headers)

print(response.text)

const options = {
  method: 'POST',
  headers: {Authorization: 'Bearer <token>', 'Content-Type': 'application/json'},
  body: JSON.stringify({
    enabled: true,
    max_price_usd: '<string>',
    adapter: '<string>',
    switch_off: ['<string>']
  })
};

fetch('https://console.sqwish.ai/v1/models/{model}/keep-warm', options)
  .then(res => res.json())
  .then(res => console.log(res))
  .catch(err => console.error(err));

{
  "id": "<string>",
  "status": "<string>",
  "evaluation_key": "<string>",
  "name": "<string>",
  "description": "<string>",
  "family": "<string>",
  "size_order": 123,
  "kind": "<string>",
  "lora": "<string>",
  "metrics": {},
  "max_prompt_tokens": 123,
  "max_questions": 123,
  "calibration": "<string>",
  "availability_reason": "<string>",
  "available": true,
  "access_requested": true,
  "adapter_capacity": {
    "max_rank": 123,
    "slots": 123
  },
  "created_at": 123,
  "deleted_at": 123,
  "discarded_at": 123,
  "on_at": 123,
  "keep_warm": true,
  "keep_warm_window": {
    "started_at": 123,
    "ends_at": 123,
    "reserved_usd": "<string>"
  },
  "keep_warm_stopped_at": 123,
  "used_at": 123,
  "switches_off_at": 123,
  "off_reason": "owner",
  "off_at": 123,
  "job_id": "<string>",
  "model_name": "<string>",
  "version": 123,
  "parent": "<string>",
  "residency": {
    "state": "hot",
    "pinned": true
  },
  "state": "loaded",
  "decision_price_nano": "<string>",
  "base": "<string>",
  "evaluation_set": {
    "dataset": {
      "id": "<string>",
      "name": "<string>",
      "archived_at": 123,
      "deleted": true
    },
    "training_examples": 123,
    "validation_examples": 123,
    "remembered_examples": 123
  },
  "request_latency": {
    "median_ms": 123,
    "samples": 123,
    "median_is_lower_bound": true,
    "start": "<string>",
    "end": "<string>"
  }
}

{
  "error": {
    "code": "<string>",
    "message": "<string>",
    "retryable": true,
    "request_id": "<string>",
    "details": [
      {
        "path": [
          "<string>"
        ],
        "message": "<string>",
        "type": "<string>"
      }
    ]
  }
}

{
  "error": {
    "code": "<string>",
    "message": "<string>",
    "retryable": true,
    "request_id": "<string>",
    "details": [
      {
        "path": [
          "<string>"
        ],
        "message": "<string>",
        "type": "<string>"
      }
    ]
  }
}

Authorizations

Authorization

string

header

required

Create an API key in the console. Required when accounts are enabled.

Path Parameters

model

string

required

Body

application/json

enabled

boolean

required

max_price_usd

string | null

The most you agree to pay for the first hour, which switching keep warm on holds, in USD, such as the price you were shown. If it costs more when it is admitted, it is refused with price_changed and nothing is held.

Pattern: ^\d{1,9}(\.\d{1,9})?$

adapter

string

The id of the model the caller saw: refused unless the model called is still that one.

Maximum string length: 140

switch_off

string[]

Families, such as router, and models, such as router@2, whose keep warm stops in the same step, to make room when the account is at its limit (warm_limit) or the model's size has no kept-warm place left (warm_capacity). Their switches stay on.

Maximum array length: 20

Response

Successful Response

A base, an early-access catalogue entry, or a trained adapter.

string

required

status

string

required

evaluation_key

string

required

name

string

description

string

family

string

size_order

integer

kind

string

lora

string | null

metrics

Metrics · object | null

max_prompt_tokens

integer

max_questions

integer

calibration

string

availability_reason

string

Why it can't answer now: no_engine (its pool has no engine right now), not_served (this server doesn't serve it), or, for an adapter, base_mismatch, engine_unconfigured or engine_changed.

available

boolean

access_requested

boolean

adapter_capacity

AdapterCapacity · object | null

Show child attributes

created_at

number

deleted_at

number

When its owner deleted it. A deleted model answers and runs nothing.

discarded_at

number

When a rollback of its family discarded it. A discarded model answers and runs nothing.

on_at

number | null

When its own switch went on: it answers calls to its number, such as router@4. Null while it is off.

keep_warm

boolean

Whether it is kept warm on its own switch. A family's keep warm doesn't set it: see keep_warm_window.

keep_warm_window

KeepWarmWindow · object | null

The hour of keep warm being billed for it, through its family or its own switch, or null when nothing is billed.

Show child attributes

keep_warm_stopped_at

number | null

When its own keep warm switched off because the next hour couldn't be reserved, while it stays off. Otherwise null.

used_at

number | null

Its idle clock: the latest of its last production call by an API key that named it, its last evaluation that named it, and when it was switched on.

switches_off_at

number | null

When idling switches it off unless it's used. Null while it is off or kept warm on its own switch, and when idling never switches anything off. A family's is also null while it serves no model.

off_reason

enum<string> | null

Why it is off: owner when it was switched off by hand, idle when nothing used it for too long. Null otherwise.

Available options:

owner,

idle

off_at

number | null

When it went off, with off_reason. Null otherwise.

job_id

string

model_name

string | null

version

integer

parent

string | null

residency

Residency · object

Show child attributes

state

enum<string> | null

Available options:

loaded,

loading,

asleep

decision_price_nano

string | null

Exact integer string of nano-USD per billed input token from the current billing policy. Zero means free; null means unpriced.

base

string

An adapter's base model id: the base whose prices it pays.

evaluation_set

EvaluationSet · object | null

Show child attributes

request_latency

RequestLatency · object

Show child attributes

Keep a model warm

curl --request POST \
  --url https://console.sqwish.ai/v1/models/{model}/keep-warm \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '
{
  "enabled": true,
  "max_price_usd": "<string>",
  "adapter": "<string>",
  "switch_off": [
    "<string>"
  ]
}
'

import requests

url = "https://console.sqwish.ai/v1/models/{model}/keep-warm"

payload = {
    "enabled": True,
    "max_price_usd": "<string>",
    "adapter": "<string>",
    "switch_off": ["<string>"]
}
headers = {
    "Authorization": "Bearer <token>",
    "Content-Type": "application/json"
}

response = requests.post(url, json=payload, headers=headers)

print(response.text)

const options = {
  method: 'POST',
  headers: {Authorization: 'Bearer <token>', 'Content-Type': 'application/json'},
  body: JSON.stringify({
    enabled: true,
    max_price_usd: '<string>',
    adapter: '<string>',
    switch_off: ['<string>']
  })
};

fetch('https://console.sqwish.ai/v1/models/{model}/keep-warm', options)
  .then(res => res.json())
  .then(res => console.log(res))
  .catch(err => console.error(err));

{
  "id": "<string>",
  "status": "<string>",
  "evaluation_key": "<string>",
  "name": "<string>",
  "description": "<string>",
  "family": "<string>",
  "size_order": 123,
  "kind": "<string>",
  "lora": "<string>",
  "metrics": {},
  "max_prompt_tokens": 123,
  "max_questions": 123,
  "calibration": "<string>",
  "availability_reason": "<string>",
  "available": true,
  "access_requested": true,
  "adapter_capacity": {
    "max_rank": 123,
    "slots": 123
  },
  "created_at": 123,
  "deleted_at": 123,
  "discarded_at": 123,
  "on_at": 123,
  "keep_warm": true,
  "keep_warm_window": {
    "started_at": 123,
    "ends_at": 123,
    "reserved_usd": "<string>"
  },
  "keep_warm_stopped_at": 123,
  "used_at": 123,
  "switches_off_at": 123,
  "off_reason": "owner",
  "off_at": 123,
  "job_id": "<string>",
  "model_name": "<string>",
  "version": 123,
  "parent": "<string>",
  "residency": {
    "state": "hot",
    "pinned": true
  },
  "state": "loaded",
  "decision_price_nano": "<string>",
  "base": "<string>",
  "evaluation_set": {
    "dataset": {
      "id": "<string>",
      "name": "<string>",
      "archived_at": 123,
      "deleted": true
    },
    "training_examples": 123,
    "validation_examples": 123,
    "remembered_examples": 123
  },
  "request_latency": {
    "median_ms": 123,
    "samples": 123,
    "median_is_lower_bound": true,
    "start": "<string>",
    "end": "<string>"
  }
}

{
  "error": {
    "code": "<string>",
    "message": "<string>",
    "retryable": true,
    "request_id": "<string>",
    "details": [
      {
        "path": [
          "<string>"
        ],
        "message": "<string>",
        "type": "<string>"
      }
    ]
  }
}

{
  "error": {
    "code": "<string>",
    "message": "<string>",
    "retryable": true,
    "request_id": "<string>",
    "details": [
      {
        "path": [
          "<string>"
        ],
        "message": "<string>",
        "type": "<string>"
      }
    ]
  }
}

Overview

Decisions

Models

Families

Datasets

Evaluations

Fine-tuning

Prompt tuning

Labeling

Accounts

Billing

Integrations

System

Keep a model warm

Authorizations

Path Parameters

Body

Response