Skip to main content
A family is a line of models for one task, such as support-router. Each model in it was trained from the one before, on one dataset that grows in versions beside the line. A model is always called by its number, such as support-router@3. Only a model that passed its gate gets a number. Most of the time you need five things:
  • Call the family by its name, support-router. It answers with its latest model that passed its gate.
  • Continue the family when you have new labelled rows. Its next model joins only if it passes its gate. The family then moves to it by itself.
  • Read the family to see the model that answers, served, and the newest one, latest.
  • Roll the family back if a new model does worse for you. This discards every newer model.
  • Switch off what you no longer call, to free a place for another model.
Switch, roll back and delete covers every state, check and error of those operations. Keep a model warm covers keeping a model loaded.

How a family grows

  1. Build a dataset, then start a family with its first fine-tune. The family is bound to that dataset for good.
  2. If the model passes its gate, it joins the family as support-router@1. A model that fails its gate joins nothing and answers nothing.
  3. A family that is switched on moves to each model that joins.
  4. When you have more labelled rows, add them as the dataset’s next version. Then continue the family. The next model trains from the latest one, on the new rows with earlier ones mixed in.
A family continues only on a dataset version newer than the one its last finished job trained on.

Read a family

Read a family to see where it stands:
GET /v1/families lists every family, without jobs.

Call a family or a model

Put the family’s name, or a model’s number, in model. Send the body to /v1/decide with your key:
An answer reports the name you called. It also reports, as version, the exact model that answered, such as support-router@3. During an evaluation, set fallback: "none" if another model’s answer would spoil the comparison.
  • Call a model through its family or by its number, never by its adapter ID. An adapter ID gets 422 invalid_reference. So does anything after @ but a number, such as support-router@latest. Either could name another model by the time a retry arrives.
  • A model that failed its gate has no number and answers nothing.
  • A family serves another model only when it moves to it.
  • Switching a model on loads it. A model that isn’t kept warm can be unloaded to make room for others. Its next call loads it again, which can take several seconds. If the load takes over 10 seconds, a base model answers, its own size first, and fallback says so. With "fallback": "none", the call gets 503 model_warming instead (cold starts).

How a family moves

A family that is switched on serves its latest model. That is the highest-numbered one that isn’t deleted or discarded. The gate decides whether a model joins, so a move doesn’t judge the model again.
  1. A fine-tune passes its gate, and its model joins the family as the next number. A model that fails its gate joins nothing, and its job’s /events show gate_failed.
  2. The server checks regularly for a newer model. It loads the model and checks that it answers, wherever the family’s current model is loaded. For a family kept warm, it does this everywhere the model’s size is served. So calls after the move find the model loaded.
  3. The family moves.
A slow load in one place doesn’t hold the move up, once the model is loaded elsewhere. GET /v1/families/support-router shows a move the family waits to make as move, such as {"to": "support-router@5", "waiting_for": "place", "reason": "live_limit"}. move is null while the family serves its latest model, or is switched off. Once a family has moved by itself, moved says when, as at. It also gives the model the family served before, as from, or null for none. moved stays until the family serves another model, such as after a rollback. From the SDKs, families.wait(family, model) reads this until the family serves model (wait for a job or a family).

What a move needs

  • A place. A move takes one only in two cases. Either the model the family served keeps answering by its own switch, or the family served nothing before. With no place free, the family keeps serving its current model and loads nothing. It moves at a later check, once a place frees.
  • A warm place, for a family kept warm. Keep warm follows the family to its new model, and so does the hour being billed. A move takes a warm place only when the model the family served stays warm on its own keep warm. With no warm place free on your account, or no kept-warm place free on the model’s size, the family waits. It keeps its keep warm, and nothing is switched off to make room.
  • A model that loads. A model that fails to load, or fails its check, leaves the family where it was. After an outage that passes, the next check tries again. After any other failure, such as a refused load, the family waits longer before each try. The wait ends early when we add or restart a server that runs models. The family keeps answering meanwhile.
A family that is switched off doesn’t move. Switching it on serves its latest model, loaded and checked first. If that model fails to load and the family served another, it switches on with that one. It then moves to the latest at a later check. Otherwise it stays off, and the switch answers with the load’s error. A family moves only to a newer model. A model it has left keeps answering its number, such as support-router@2, while its own switch is on. To stop a family answering, switch it off. To take it back to an older model, roll it back.

Emails

We email you once for each model a family moves to by itself. We also email once for each model whose move waits: A rollback sends no email of its own. If its move waits, you get the waiting email, the same as for any move.