> For the complete documentation index, see [llms.txt](https://docs.layerlens.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.layerlens.ai/13.6-system-judges-implementation/system.md).

# System judge implementation reference

Implementation reference for Stratix system judges — record shape, scoring contracts, and how they run.

This is the implementation-level companion to the [System judges](/8.-evaluate-score-the-outputs/system-judges.md) catalog. The catalog page is the "what and when"; this section is the "how": the exact record shape, the scoring contract each judge returns, and how a judge is stored and executed.

## Record shape

Every system judge is a row in the platform `judges` collection with the same fields as a custom judge, plus the system markers:

| Field             | Meaning                                                                       |
| ----------------- | ----------------------------------------------------------------------------- |
| `name`            | Human-readable judge name (e.g. `Faithfulness`)                               |
| `evaluation_goal` | The natural-language rubric the judge LLM is instructed with                  |
| `model_id`        | The judge model (system judges default to a frontier Claude Opus-class model) |
| `is_system`       | `true` for LayerLens-shipped judges                                           |
| `organization_id` | `null` for system judges; set for cloned/custom judges                        |
| `inputs`          | Template variables the rubric expects (e.g. `{{output}}`, `{{context}}`)      |

System judges are **read-only** records — customization is clone-then-edit, and tuning is done with [GEPA optimization](/9.-improve-tune-the-system/judge-optimization.md) against your own labeled set. See the catalog page for the clone workflow.

## Scoring contracts

System judges return one of three shapes, depending on the dimension:

* **Scalar** — a `0.0–1.0` score plus a structured detail list. Faithfulness (unsupported claims), Citation Accuracy (citation failures).
* **Categorical verdict** — one of a fixed enum. Refusal Quality (`appropriate_answer` / `appropriate_refusal` / `over_refusal` / `under_refusal`), Hallucination Detector severity (`none` / `minor` / `material` / `dangerous`).
* **Sub-scored** — an overall score with named sub-scores. Tone & Register (warmth / formality / brevity / jargon), Explanation Quality (clarity / completeness / traceability).

The scalar-with-detail contract is documented end to end in the worked example:

* [Faithfulness — implementation reference](/13.6-system-judges-implementation/faithfulness-implementation.md)

## All system judges — implementation contracts

Every system judge in the [catalog](/8.-evaluate-score-the-outputs/system-judges.md) and its implementation-level contract (template inputs + return shape). All share the same record fields (`name`, `evaluation_goal`, `model_id`, `is_system: true`) and run on the same LLM-as-Judge engine.

### General system judges

| Judge                  | Inputs                                                                    | Output contract                                                                                |
| ---------------------- | ------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------- |
| Faithfulness           | `{{output}}`, `{{context}}`                                               | scalar `0.0–1.0` + `unsupported_claims[]`                                                      |
| Hallucination Detector | `{{output}}`, `{{context}}`?                                              | severity `none` / `minor` / `material` / `dangerous` + flagged claims                          |
| Refusal Quality        | `{{prompt}}`, `{{output}}`, `{{context}}`?, `{{scope}}`?                  | verdict `appropriate_answer` / `appropriate_refusal` / `over_refusal` / `under_refusal`        |
| Citation Accuracy      | `{{output}}`, `{{context}}`                                               | scalar `0.0–1.0` + failures `not_in_source` / `misquoted` / `wrong_section`                    |
| Tone & Register        | `{{output}}`, `{{expected_register}}`                                     | overall + sub-scores: warmth / formality / brevity / jargon                                    |
| De-escalation Quality  | `{{context}}`, `{{output}}`                                               | overall + acknowledgement / pathway / tone (failure sub-booleans)                              |
| Plain-language Quality | `{{output}}`, `{{audience}}`                                              | scalar + concept-unpacking / actionability / jargon (pair with the Flesch-Kincaid code grader) |
| Explanation Quality    | `{{decision}}`, `{{output}}`, `{{reviewer_profile}}`                      | sub-scores: clarity / completeness / traceability                                              |
| Multilingual Parity    | `{{source_output}}`, `{{output}}`, `{{target_language}}`, `{{glossary}}`? | parity scalar (accuracy / tone / required disclosures preserved)                               |
| Editorial Judgment     | `{{context}}`, `{{output}}`                                               | framing / attribution / headline-alignment + inserted-bias flag                                |

### Industry-specialty system judges

Same record fields and clone-and-tune workflow; these target high-stakes verticals.

| Judge                          | Industry           | What the verdict asserts                                                         |
| ------------------------------ | ------------------ | -------------------------------------------------------------------------------- |
| Contract Clause Interpretation | Legal, real estate | Interpretation matches the clause's plain meaning + governing law                |
| Mata Citation Verifier         | Legal              | Each filed citation exists, is good law, and supports the claim                  |
| Privilege Leak Detector        | Legal              | Output does not leak attorney-client privileged content                          |
| Clinical Reasoning Soundness   | Healthcare         | Differential diagnosis appropriate; red-flag features identified                 |
| Medical-coding Justification   | Healthcare         | ICD-10-CM / CPT / HCPCS codes supported by documentation; no unbundling/upcoding |
| Switching-step Faithfulness    | Energy & utilities | Grid switching sequence matches approved procedure + safety preconditions        |
| Network Mitigation Rationale   | Telecom, energy    | Recommended mitigation is grounded in the runbook                                |
| Fare-rule Paraphrase Quality   | Travel             | Paraphrase preserves conditions, dollar amounts, and time windows                |
| Steering-language Detector     | Real estate        | Listing copy does not steer on a protected-class basis                           |
| Pedagogical Scaffolding        | Education          | Tutor guides the learner rather than giving the answer outright                  |

For the full worked rubric (bands, tuning, run example), see [Faithfulness — implementation reference](/13.6-system-judges-implementation/faithfulness-implementation.md); the other scalars follow the same band structure, and the categorical judges return the enum shown above.

## How a judge runs

A judge is executed by the platform's LLM-as-Judge engine: the `evaluation_goal` rubric is rendered with the invocation's `inputs`, sent to the judge `model_id`, and the structured verdict is persisted against the trace or evaluation. It is the same engine whether the judge is a system judge or one of your own.

* **Dashboard**: Judges → filter Type: System → open a judge to read its rubric.
* **SDK / REST**: list and run judges programmatically.

```python
from layerlens import Stratix
client = Stratix()

# Discover system judges
for j in client.judges.get_many(type="system"):
    print(j.id, j.name, j.model_id)

# Run one against a trace
result = client.trace_evaluations.create(trace_id=trace_id, judge_id=judge_id)
```

## Reference

* [Judges — API reference](https://github.com/LayerLens/gitbook-full/blob/main/13-reference/api/judges.md)
* [Judges — SDK reference](https://github.com/LayerLens/gitbook-full/blob/main/13-reference/sdk-python/judges.md)
* [System judges — catalog](/8.-evaluate-score-the-outputs/system-judges.md)
* [Judge Optimization (GEPA)](/9.-improve-tune-the-system/judge-optimization.md)