Recipe — confidence-weighted aggregation for noisy judges.
When your judge returns a confidence score along with the verdict, weight by confidence:
agg = sum(v * c for v, c in zip(verdicts, confidences)) / sum(confidences)
Use sparingly — confidence-weighting hides low-quality judges instead of fixing them. GEPA-optimize first.
Concept: Judges
Last updated 1 month ago
Was this helpful?
