support-router. Each model in it was trained
from the one before, on one dataset that grows in versions beside the line.
A model is always called by its number, such as support-router@3. Only a model that passed its
gate gets a number.
Most of the time you need five things:
- Call the family by its name,
support-router. It answers with its latest model that passed its gate. - Continue the family when you have new labelled rows. Its next model joins only if it passes its gate. The family then moves to it by itself.
- Read the family to see the model that answers,
served, and the newest one,latest. - Roll the family back if a new model does worse for you. This discards every newer model.
- Switch off what you no longer call, to free a place for another model.
How a family grows
- Build a dataset, then start a family with its first fine-tune. The family is bound to that dataset for good.
- If the model passes its gate, it joins the family as
support-router@1. A model that fails its gate joins nothing and answers nothing. - A family that is switched on moves to each model that joins.
- When you have more labelled rows, add them as the dataset’s next version. Then continue the family. The next model trains from the latest one, on the new rows with earlier ones mixed in.
Read a family
Read a family to see where it stands:GET /v1/families lists every family, without jobs.
Call a family or a model
Put the family’s name, or a model’s number, inmodel. Send the body to /v1/decide with your key:
An answer reports the name you called. It also reports, as
version, the exact model that
answered, such as support-router@3. During an evaluation, set fallback: "none" if another
model’s answer would spoil the comparison.
- Call a model through its family or by its number, never by its adapter ID. An adapter ID gets
422 invalid_reference. So does anything after@but a number, such assupport-router@latest. Either could name another model by the time a retry arrives. - A model that failed its gate has no number and answers nothing.
- A family serves another model only when it moves to it.
- Switching a model on loads it. A model that isn’t kept warm can be unloaded
to make room for others. Its next call loads it again, which can take several seconds. If the load
takes over 10 seconds, a base model answers, its own size first, and
fallbacksays so. With"fallback": "none", the call gets503 model_warminginstead (cold starts).
How a family moves
A family that is switched on serves its latest model. That is the highest-numbered one that isn’t deleted or discarded. The gate decides whether a model joins, so a move doesn’t judge the model again.- A fine-tune passes its gate, and its model joins the family as the next number. A model that
fails its gate joins nothing, and its job’s
/eventsshowgate_failed. - The server checks regularly for a newer model. It loads the model and checks that it answers, wherever the family’s current model is loaded. For a family kept warm, it does this everywhere the model’s size is served. So calls after the move find the model loaded.
- The family moves.
GET /v1/families/support-router shows a move the family waits to make as move, such as
{"to": "support-router@5", "waiting_for": "place", "reason": "live_limit"}. move is null
while the family serves its latest model, or is switched off.
Once a family has moved by itself,
moved says when, as at. It also gives the model the family
served before, as from, or null for none. moved stays until the family serves another model,
such as after a rollback.
From the SDKs, families.wait(family, model) reads this until the family serves model
(wait for a job or a family).
What a move needs
- A place. A move takes one only in two cases. Either the model the family served keeps answering by its own switch, or the family served nothing before. With no place free, the family keeps serving its current model and loads nothing. It moves at a later check, once a place frees.
- A warm place, for a family kept warm. Keep warm follows the family to its new model, and so does the hour being billed. A move takes a warm place only when the model the family served stays warm on its own keep warm. With no warm place free on your account, or no kept-warm place free on the model’s size, the family waits. It keeps its keep warm, and nothing is switched off to make room.
- A model that loads. A model that fails to load, or fails its check, leaves the family where it was. After an outage that passes, the next check tries again. After any other failure, such as a refused load, the family waits longer before each try. The wait ends early when we add or restart a server that runs models. The family keeps answering meanwhile.
support-router@2, while its own switch is on. To stop a family answering,
switch it off. To take it back to an older
model, roll it back.
Emails
We email you once for each model a family moves to by itself. We also email once for each model whose move waits:
A rollback sends no email of its own. If its move waits, you get the waiting email, the same as
for any move.
