> For the complete documentation index, see [llms.txt](https://docs.layerlens.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.layerlens.ai/8.-evaluate-score-the-outputs/judges.md).

# Judges

Build, test, and manage LLM-as-a-judge graders in Stratix Premium.

{% hint style="info" %}
**Available in Stratix Premium.** This surface is part of the logged-in workspace at [stratix.layerlens.ai](https://stratix.layerlens.ai). Stratix Public users can browse the catalog but cannot use this feature.
{% endhint %}

A **judge** is an LLM that grades an output along a subjective dimension — helpfulness, faithfulness, tone, safety. Stratix Premium ships with system judges, lets you build custom judges, and tunes them with GEPA.

URL: [`stratix.layerlens.ai/dashboard/agent-evaluation/judges`](https://stratix.layerlens.ai/dashboard/agent-evaluation/judges)

## What you can do

* Browse system judges (vendored, ready-to-use)
* Browse your org's custom judges
* Create a new custom judge
* Test a judge against sample outputs
* GEPA-optimize a judge against labeled examples
* Apply a judge to evaluations and trace evaluations

## System judges

LayerLens ships a small library of system judges for common dimensions:

* Helpfulness
* Faithfulness (RAG-shaped)
* Safety
* Tone-appropriateness
* Brevity / verbosity
* Structured-output validity

Use these as starting points; clone and customize for your team's bar.

## Building a custom judge

The judge builder has 5 fields:

1. **Name and description**
2. **Output type** — binary, scored (1-5), labeled (multi-class)
3. **Judging model** — the LLM that runs the rubric
4. **Rubric** — the prompt
5. **Test examples** — paste samples to validate

A good rubric:

* Names the dimension explicitly
* Shows examples of "good" with labels
* Shows examples of "bad" with labels
* Specifies the output format

## GEPA judge optimization

Once you have ≥30 labeled examples, run **GEPA optimization**. Stratix tunes the rubric prompt to maximize agreement with your labels. See [Judge Optimization (GEPA)](/9.-improve-tune-the-system/judge-optimization.md).

## Applying judges

Judges are reusable across:

* Evaluations (private and shared)
* Trace evaluations
* Agentic evaluations

In any of those flows, the scoring step lets you stack any number of judges alongside scorers.

## Where to next

* [Judge Optimization (GEPA)](/9.-improve-tune-the-system/judge-optimization.md)
* [Concept: Judges](/8.-evaluate-score-the-outputs/judges-1.md)
* [First judge](/2.-get-started/first-judge.md)
* [Tutorial: Build your first judge](/8.-evaluate-score-the-outputs/02-first-judge.md)