Evaluations
Retry an evaluation
POST
/
v1
/
evaluations
/
{evaluation_id}
/
retry
Retry an evaluation
curl --request POST \
--url https://api.sqwish.ai/v1/evaluations/{evaluation_id}/retry \
--header 'Authorization: Bearer <token>'import requests
url = "https://api.sqwish.ai/v1/evaluations/{evaluation_id}/retry"
headers = {"Authorization": "Bearer <token>"}
response = requests.post(url, headers=headers)
print(response.text)const options = {method: 'POST', headers: {Authorization: 'Bearer <token>'}};
fetch('https://api.sqwish.ai/v1/evaluations/{evaluation_id}/retry', options)
.then(res => res.json())
.then(res => console.log(res))
.catch(err => console.error(err));{
"id": "<string>",
"dataset_id": "<string>",
"dataset_version": 123,
"model_id": "<string>",
"model_sha256": "<string>",
"model_label": "<string>",
"model_name": "<string>",
"version": 123,
"status": "queued",
"created_at": 123,
"updated_at": 123,
"generation": 123,
"training_job_id": "<string>",
"dataset_sha256": "<string>",
"evaluation_sha256": "<string>",
"method": "sft",
"evaluation_rows": 123,
"decisions": 123,
"completed_rows": 123,
"metrics": {},
"error": {},
"named": "<string>"
}{
"error": {
"code": "<string>",
"message": "<string>",
"retryable": true,
"request_id": "<string>",
"details": [
{
"path": [
"<string>"
],
"message": "<string>",
"type": "<string>"
}
]
}
}{
"error": {
"code": "<string>",
"message": "<string>",
"retryable": true,
"request_id": "<string>",
"details": [
{
"path": [
"<string>"
],
"message": "<string>",
"type": "<string>"
}
]
}
}Authorizations
Create an API key in the console.
Path Parameters
Response
Successful Response
The dataset version whose evaluation rows it scores.
Available options:
queued, running, cancelling, succeeded, failed, cancelled Available options:
sft, reward Null until the evaluation succeeds. Then n, the decisions scored, with accuracy, loss, brier and ece for expected answers or expected_reward for rewards, and latency_ms, the median server milliseconds per request, when it was measured. A result taken from a fine-tune's own held-out scoring sends no request, so it has none.
The family, or the model by its number, the request named, whose idle clock the run moves when it finishes.
Retry an evaluation
curl --request POST \
--url https://api.sqwish.ai/v1/evaluations/{evaluation_id}/retry \
--header 'Authorization: Bearer <token>'import requests
url = "https://api.sqwish.ai/v1/evaluations/{evaluation_id}/retry"
headers = {"Authorization": "Bearer <token>"}
response = requests.post(url, headers=headers)
print(response.text)const options = {method: 'POST', headers: {Authorization: 'Bearer <token>'}};
fetch('https://api.sqwish.ai/v1/evaluations/{evaluation_id}/retry', options)
.then(res => res.json())
.then(res => console.log(res))
.catch(err => console.error(err));{
"id": "<string>",
"dataset_id": "<string>",
"dataset_version": 123,
"model_id": "<string>",
"model_sha256": "<string>",
"model_label": "<string>",
"model_name": "<string>",
"version": 123,
"status": "queued",
"created_at": 123,
"updated_at": 123,
"generation": 123,
"training_job_id": "<string>",
"dataset_sha256": "<string>",
"evaluation_sha256": "<string>",
"method": "sft",
"evaluation_rows": 123,
"decisions": 123,
"completed_rows": 123,
"metrics": {},
"error": {},
"named": "<string>"
}{
"error": {
"code": "<string>",
"message": "<string>",
"retryable": true,
"request_id": "<string>",
"details": [
{
"path": [
"<string>"
],
"message": "<string>",
"type": "<string>"
}
]
}
}{
"error": {
"code": "<string>",
"message": "<string>",
"retryable": true,
"request_id": "<string>",
"details": [
{
"path": [
"<string>"
],
"message": "<string>",
"type": "<string>"
}
]
}
}