Recipe: build a faithfulness judge for RAG

Recipe — RAG faithfulness judge. Pointer to the SDK rag_assessment sample.

Canonical sample: samples/cowork/rag_assessment.py — RAG Runner-Quality Judge multi-agent pattern. One agent executes retrieval queries; the other evaluates groundedness and retrieval quality.

Pattern

1. Capture traces with retrieval and generation spans separately.
2. Build a faithfulness judge: client.judges.create(name=, evaluation_goal=) whose rubric checks claims against retrieved chunks.
3. GEPA-optimize against ≥50 labeled examples — see Bootstrap a judge before GEPA.
4. Apply via client.trace_evaluations.create(trace_id=, judge_id=).

See also

• Use case: RAG evaluation
• Concept: Judges
• Tutorial 5: GEPA optimize