Judge Optimization (GEPA)
GEPA judge optimization — automatically tune your judge rubric to match labeled ground truth.
Last updated
Was this helpful?
⌘Ctrlk
GEPA judge optimization — automatically tune your judge rubric to match labeled ground truth.
Available in Stratix Premium. This surface is part of the logged-in workspace at stratix.layerlens.ai. Stratix Public users can browse the catalog but cannot use this feature.
GEPA optimization tunes a judge's rubric prompt against a labeled ground-truth set. The result is a rubric that better matches your team's actual quality bar — typically a 10-20 percentage-point lift in human-agreement rate.
GEPA explores prompt variations, runs each against your labeled examples, and picks the variation with the highest agreement rate.
A built judge (see Judges)
≥30 labeled examples — input/output pairs with the human verdict you'd want the judge to produce
ECU credits to run the optimization
Open the judge in Premium → Agent Evaluation → Judges
Click Optimize
Upload or select your labeled examples
Configure: number of iterations, target metric (agreement, F1, accuracy)
Run
The optimization result shows:
Agreement rate before — how often the original rubric matched your labels
Agreement rate after — how often the optimized rubric matches
Diff — what changed in the rubric prompt
Per-iteration history — the trajectory of the optimization
When you add more labeled examples
When your team's quality bar shifts (you label things differently than before)
When the judging model is upgraded
For agentic evaluations, GEPA-optimized judges keep your subjective bar honest. Out-of-the-box judges drift; tuned judges hold tight to your labels.
Last updated
Was this helpful?
Was this helpful?
from layerlens import Stratix
client = Stratix()
opt = client.judge_optimizations.create(
judge_id="judge_abc",
labeled_examples_dataset_id="dataset_xyz",
iterations=20,
)
result = client.judge_optimizations.wait_for_completion(opt.id)
print(f"Agreement before: {result.before}, after: {result.after}")