Recipe: score retrieval quality

Recipe — score retrieval quality. Pointer to the SDK rag_assessment sample.

Canonical sample: samples/cowork/rag_assessment.py — covers retrieval-stage and generation-stage scoring separately.

Pattern

Capture retrieval span output (chunk IDs returned) alongside ground-truth chunk IDs in your trace input. Build a custom judge that scores retrieval quality (groundedness, ordering, coverage) on its own.

For deterministic precision@k or recall@k computation, score with a custom scorer via client.scorers.create(name=, description=, model_id=, prompt=).

See also

• Use case: RAG evaluation
• Concept: Scorers