Judges

client.judges — CRUD for evaluation judges, with version-on-update semantics.

Create

name and evaluation_goal are required. model_id is optional — if omitted, a default judging model is used.

from layerlens import Stratix

client = Stratix()

# Optionally fetch a model to use as the judge's LLM
models = client.models.get(type="public", name="gpt-4o")
model = models[0]

judge = client.judges.create(
 name="Code Quality Judge",
 evaluation_goal="Evaluate the quality of code output including correctness, readability, and style",
 model_id=model.id,
)
print(f"Created judge {judge.id}: {judge.name}")

Get

judge = client.judges.get(judge_id)
print(f"Judge: {judge.name}, version: {judge.version}")

List

get_many() returns a JudgesResponse with .judges, .total_count:

response = client.judges.get_many()
print(f"Found {response.total_count} judges")
for j in response.judges:
 print(f" - {j.name} (v{j.version}, {j.run_count} runs)")

Update — creates a new version

Updating a judge creates a new version. The prior version remains accessible for already-attached evaluations:

updated = client.judges.update(
 judge.id,
 name="Updated Code Quality Judge",
 evaluation_goal="Evaluate code output for correctness, readability, style, and security",
)
print(f"Updated judge {updated.id}, version {updated.version}")

Delete

deleted = client.judges.delete(judge.id)
print(f"Deleted judge {deleted.id}")

Async

from layerlens import AsyncStratix

async def example():
 client = AsyncStratix()
 judge = await client.judges.create(name="...", evaluation_goal="...")
 response = await client.judges.get_many()

Versioning model

Each update() call creates a new JudgeVersion linked to the parent judge
The judge's .version field reflects the latest version
Already-running evaluations stay pinned to the version they started with
Use JudgeSnapshot for point-in-time references in audit trails

Source samples

Sample

What it shows

samples/core/create_judge.py

Full CRUD lifecycle

samples/core/judge_creation_and_test.py

Build a custom PII detection judge with validation

samples/core/judge_optimization.py

Estimate, run, apply judge optimizations

samples/core/trace_evaluation.py

Apply judge to traces

samples/industry/healthcare_clinical.py

Domain-tuned clinical judge

Good night