D1_API_KEY as in the quickstart and run the same request:
Use the result
Readdecisions.relevance.expected_level on the recipe’s 0–3 scale and inspect the probability mass across levels. Use it to rank or filter candidates, then evaluate the resulting answers rather than only the ranking scores.
For a simple integration, make one request per passage. If you score several in one context, give each decision a unique ID and explicitly identify its passage. Stay within the model’s input and question limits.
A passage can be on topic without containing an answer. Test that distinction, along with partial answers, distracting text, duplicated passages and conflicting sources. Relevance is not source credibility or a guarantee of factual correctness.
What was evaluated
The historical result measures the labelled relevance decision. It does not measure the downstream answer improvement from your retrieval stack. Compare answer quality with and without this ranking stage on the same held-out questions.Historical evaluation on 2026-09-28, using sqwish-d1-core. These measurements describe that checkpoint and dataset, not current production performance or an accuracy guarantee.
- Dataset: MIRACL English (dev) (Apache-2.0 (the MTEB passage copy is CC BY-SA 4.0)).
- Split: dev, queries whose hash falls below 0.5.
- Sample: 1,000 rows, 1,000 measured decisions.
- Measured decision IDs:
relevance. - Wording: hand-written.
- Checkpoint SHA-256:
dd420ca652cfaa10279eabb54d15afd50fe9d24d357498c4dbc512b1c3d73bb2.
The majority-class baseline has accuracy 0.724; the class-prior baseline has log loss 0.5891.
Only the decision IDs listed above were measured. Dataset labels, class balance and wording affect these results. The intervals do not measure distribution shift. Re-evaluate with your own cases, including ambiguous and out-of-scope inputs.