Evaluate (workflow)

Stage 4 — Evaluate. Score outputs and traces with scorers and judges.

Stage 4. Evaluate runs scorers and judges over your outputs to produce verdicts and scores.

The question this stage answers

"How good is the AI output — quantitatively?"

What to do

1. Define quality dimensions. What does "good" mean for this feature? Helpfulness? Faithfulness? Tone? Structured-output validity? List them.
2. Pick scorers for objective dimensions. Exact match, regex, JSON schema, semantic similarity, etc.
3. Build judges for subjective dimensions. One per dimension. GEPA-optimize against ≥30 labeled examples.
4. Run. Standard evaluation, or trace evaluation against your trace set.
5. Read the verdict. Top-line plus per-row failures. Spend most of your time on the failures — they're where you'll improve.

What you'll have at the end

• A reusable scoring config (your scorers + judges)
• A baseline score
• A list of failed rows / failed traces with root-cause links

Pre-deploy vs continuous

• Pre-deploy — run before merging changes; gate CI
• Continuous — run on production traces on a schedule; alert on drift

Both use the same scoring config; the cadence and consumer are different.

Where to next

• Improve →
• Concept: Evaluations
• Concept: Judges
• Concept: Scorers
• Tutorial: First evaluation