Judge optimizations

client.judge_optimizations — automated judge optimization (GEPA-style). Estimate, create, get, apply.

Automated judge optimization tunes a judge's prompt against examples to improve scoring accuracy. Use the client.judge_optimizations resource.

Estimate cost

Always estimate before running:

estimate = client.judge_optimizations.estimate(
 judge_id="judge_abc",
 budget="medium", # "light" | "medium" (default) | "heavy"
)

Create (start optimization run)

optimization = client.judge_optimizations.create(
 judge_id="judge_abc",
 budget="medium",
)
# optimization.status starts as "pending" / "in_progress"

Poll

import time
while True:
 optimization = client.judge_optimizations.get(optimization.id)
 if optimization.status.value in ("success", "failure"):
 break
 time.sleep(30)

Apply (commits optimized prompt)

if optimization.status.value == "success":
 print(f"Baseline: {optimization.baseline_accuracy}")
 print(f"Optimized: {optimization.optimized_accuracy}")
 client.judge_optimizations.apply(optimization.id)
 # Creates a new judge version with the optimized goal

List

response = client.judge_optimizations.get_many(judge_id="judge_abc", page=1, page_size=20)

Object fields

JudgeOptimizationRun includes:

status — OptimizationRunStatus enum (pending, in_progress, success, failure)
baseline_accuracy — measured before optimization
optimized_accuracy — measured after
original_goal — original judge evaluation_goal
optimized_goal — new prompt
estimated_cost / actual_cost
applied_version — judge version ID once applied

Budget levels

Budget

When to use

light

Quick validation, ≤30 labeled examples

medium (default)

Standard production tuning, 30-100 labels

heavy

Maximum exploration, 100+ labels, high-value judges

Good morning