> For the complete documentation index, see [llms.txt](https://docs.layerlens.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.layerlens.ai/13.6-system-judges-implementation/system.md).

# System judge implementation reference

This is the implementation-level companion to the [System judges](/8.-evaluate-score-the-outputs/system-judges.md) catalog. The catalog page is the "what and when"; this section is the "how": the exact record shape, the scoring contract each judge returns, and how a judge is stored and executed.

## Record shape

Every system judge is a row in the platform `judges` collection with the same fields as a custom judge, plus the system markers:

| Field             | Meaning                                                                       |
| ----------------- | ----------------------------------------------------------------------------- |
| `name`            | Human-readable judge name (e.g. `Faithfulness`)                               |
| `evaluation_goal` | The natural-language rubric the judge LLM is instructed with                  |
| `model_id`        | The judge model (system judges default to a frontier Claude Opus-class model) |
| `is_system`       | `true` for LayerLens-shipped judges                                           |
| `organization_id` | `null` for system judges; set for cloned/custom judges                        |
| `inputs`          | Template variables the rubric expects (e.g. `{{output}}`, `{{context}}`)      |

System judges are **read-only** records — customization is clone-then-edit, and tuning is done with [GEPA optimization](/9.-improve-tune-the-system/judge-optimization.md) against your own labeled set. See the catalog page for the clone workflow.

## Scoring contracts

System judges return one of three shapes, depending on the dimension:

* **Scalar** — a `0.0–1.0` score plus a structured detail list. Faithfulness (unsupported claims), Citation Accuracy (citation failures).
* **Categorical verdict** — one of a fixed enum. Refusal Quality (`appropriate_answer` / `appropriate_refusal` / `over_refusal` / `under_refusal`), Hallucination Detector severity (`none` / `minor` / `material` / `dangerous`).
* **Sub-scored** — an overall score with named sub-scores. Tone & Register (warmth / formality / brevity / jargon), Explanation Quality (clarity / completeness / traceability).

The scalar-with-detail contract is documented end to end in the worked example:

* [Faithfulness — implementation reference](/13.6-system-judges-implementation/faithfulness-implementation.md)

## All system judges — implementation contracts

Every system judge in the [catalog](/8.-evaluate-score-the-outputs/system-judges.md) and its implementation-level contract (template inputs + return shape). All share the same record fields (`name`, `evaluation_goal`, `model_id`, `is_system: true`) and run on the same LLM-as-Judge engine.

### General system judges

| Judge                  | Inputs                                                                    | Output contract                                                                                |
| ---------------------- | ------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------- |
| Faithfulness           | `{{output}}`, `{{context}}`                                               | scalar `0.0–1.0` + `unsupported_claims[]`                                                      |
| Hallucination Detector | `{{output}}`, `{{context}}`?                                              | severity `none` / `minor` / `material` / `dangerous` + flagged claims                          |
| Refusal Quality        | `{{prompt}}`, `{{output}}`, `{{context}}`?, `{{scope}}`?                  | verdict `appropriate_answer` / `appropriate_refusal` / `over_refusal` / `under_refusal`        |
| Citation Accuracy      | `{{output}}`, `{{context}}`                                               | scalar `0.0–1.0` + failures `not_in_source` / `misquoted` / `wrong_section`                    |
| Tone & Register        | `{{output}}`, `{{expected_register}}`                                     | overall + sub-scores: warmth / formality / brevity / jargon                                    |
| De-escalation Quality  | `{{context}}`, `{{output}}`                                               | overall + acknowledgement / pathway / tone (failure sub-booleans)                              |
| Plain-language Quality | `{{output}}`, `{{audience}}`                                              | scalar + concept-unpacking / actionability / jargon (pair with the Flesch-Kincaid code grader) |
| Explanation Quality    | `{{decision}}`, `{{output}}`, `{{reviewer_profile}}`                      | sub-scores: clarity / completeness / traceability                                              |
| Multilingual Parity    | `{{source_output}}`, `{{output}}`, `{{target_language}}`, `{{glossary}}`? | parity scalar (accuracy / tone / required disclosures preserved)                               |
| Editorial Judgment     | `{{context}}`, `{{output}}`                                               | framing / attribution / headline-alignment + inserted-bias flag                                |

### Industry-specialty system judges

Same record fields and clone-and-tune workflow; these target high-stakes verticals.

| Judge                          | Industry           | What the verdict asserts                                                         |
| ------------------------------ | ------------------ | -------------------------------------------------------------------------------- |
| Contract Clause Interpretation | Legal, real estate | Interpretation matches the clause's plain meaning + governing law                |
| Mata Citation Verifier         | Legal              | Each filed citation exists, is good law, and supports the claim                  |
| Privilege Leak Detector        | Legal              | Output does not leak attorney-client privileged content                          |
| Clinical Reasoning Soundness   | Healthcare         | Differential diagnosis appropriate; red-flag features identified                 |
| Medical-coding Justification   | Healthcare         | ICD-10-CM / CPT / HCPCS codes supported by documentation; no unbundling/upcoding |
| Switching-step Faithfulness    | Energy & utilities | Grid switching sequence matches approved procedure + safety preconditions        |
| Network Mitigation Rationale   | Telecom, energy    | Recommended mitigation is grounded in the runbook                                |
| Fare-rule Paraphrase Quality   | Travel             | Paraphrase preserves conditions, dollar amounts, and time windows                |
| Steering-language Detector     | Real estate        | Listing copy does not steer on a protected-class basis                           |
| Pedagogical Scaffolding        | Education          | Tutor guides the learner rather than giving the answer outright                  |

For the full worked rubric (bands, tuning, run example), see [Faithfulness — implementation reference](/13.6-system-judges-implementation/faithfulness-implementation.md); the other scalars follow the same band structure, and the categorical judges return the enum shown above.

## How a judge runs

A judge is executed by the platform's LLM-as-Judge engine: the `evaluation_goal` rubric is rendered with the invocation's `inputs`, sent to the judge `model_id`, and the structured verdict is persisted against the trace or evaluation. It is the same engine whether the judge is a system judge or one of your own.

* **Dashboard**: Judges → filter Type: System → open a judge to read its rubric.
* **SDK / REST**: list and run judges programmatically.

```python
from layerlens import Stratix
client = Stratix()

# Discover system judges
for j in client.judges.get_many(type="system"):
    print(j.id, j.name, j.model_id)

# Run one against a trace
result = client.trace_evaluations.create(trace_id=trace_id, judge_id=judge_id)
```

## Reference

* [Judges — API reference](https://github.com/LayerLens/gitbook-full/blob/main/13-reference/api/judges.md)
* [Judges — SDK reference](https://github.com/LayerLens/gitbook-full/blob/main/13-reference/sdk-python/judges.md)
* [System judges — catalog](/8.-evaluate-score-the-outputs/system-judges.md)
* [Judge Optimization (GEPA)](/9.-improve-tune-the-system/judge-optimization.md)


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.layerlens.ai/13.6-system-judges-implementation/system.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.