> For the complete documentation index, see [llms.txt](https://docs.layerlens.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.layerlens.ai/8.-evaluate-score-the-outputs/scorers-1.md).

# Scorers

LLM-backed scorers in Stratix — model + prompt, when to use them, when to use a judge instead.

A **scorer** in Stratix is an **LLM-backed grader** — a model plus an evaluation prompt — applied to an evaluation row. Scorers and judges share the LLM-evaluation surface; the two are separated by **lifecycle and usage**, not by implementation kind.

|                | Scorer                                                 | Judge                                         |
| -------------- | ------------------------------------------------------ | --------------------------------------------- |
| Implementation | LLM (model + prompt)                                   | LLM (model + rubric)                          |
| Versioning     | Immutable                                              | Versioned with execution history              |
| Where it runs  | Inside an evaluation run (benchmark or custom dataset) | Standalone, against traces or evaluation runs |
| Optimization   | n/a                                                    | GEPA-tunable against labeled examples         |
| Common use     | Reusable rubric you apply across many evaluations      | Subjective dimension you tune over time       |

Both call an LLM with a prompt and return a score; the distinction is "where in the workflow does it live."

## Anatomy of a scorer

A scorer is a record with:

* **Name** (3–64 chars) and **description** (10–500 chars)
* **Model** — the LLM that runs the prompt
* **Prompt** — the evaluation instructions, including how to interpret inputs and return a score
* Optional **organization\_id** / **project\_id** for tenant-scoped scorers

## When to author a scorer

* You have a scoring rubric you want to **reuse across many benchmarks** in your org.
* The rubric is stable — you don't expect to iterate on it with labeled examples.
* You want to apply it as part of a benchmark evaluation, not against traces.

When the rubric needs labeled-example tuning, version history, or trace-level evaluation, **use a** [**judge**](/8.-evaluate-score-the-outputs/judges-1.md) **instead.**

## Authoring patterns

**Rubric-style prompt**

```
Rate the OUTPUT on a 1–5 scale for factual accuracy against the EXPECTED answer.
5 = every claim correct.
3 = mostly correct with one minor error.
1 = materially wrong.
Return only the integer.

EXPECTED: {{expected}}
OUTPUT: {{output}}
```

**Pass/fail prompt**

```
Does OUTPUT contain a citation that exists in CONTEXT?
Return "yes" or "no" only.

CONTEXT: {{context}}
OUTPUT: {{output}}
```

**Structured-judgment prompt**

```
Evaluate OUTPUT and return a JSON object:
{
 "score": <0–1>,
 "violations": [...],
 "rationale": "..."
}

INPUT: {{input}}
OUTPUT: {{output}}
```

## Scorer composition in evaluations

An evaluation can stack multiple scorers. Each produces a per-row verdict; the evaluation's overall score is configurable:

* **All-pass** — every scorer must pass
* **Any-pass** — at least one scorer must pass
* **Mean** — average score across scorers
* **Custom weighting** — weighted aggregation

## Org-scoped library

Org-scoped scorers are reusable across every benchmark in your workspace. System scorers shipped by LayerLens are available across all orgs.

## Deterministic / code-based graders

Some evaluation needs are inherently deterministic — Flesch-Kincaid grade level, JSON-schema validity, regex match, statistical fairness ratios, citation-existence database lookups. These do **not** fit the Scorer model (which is LLM-prompt-driven). Treat them as separate **code graders**, computed by the evaluation runtime independently from scorers. See [Custom code grader](https://github.com/LayerLens/gitbook-full/blob/main/08-evaluate/cookbook/custom-code-scorer.md) for the pattern.

## Where to next

* [Judges](/8.-evaluate-score-the-outputs/judges-1.md) — for tunable, versioned, trace-level evaluation
* [Stratix Premium — Scorers](/8.-evaluate-score-the-outputs/scorers.md)
* [Evaluations](/8.-evaluate-score-the-outputs/evaluations-1.md)
* [SDK reference — Scorers](/8.-evaluate-score-the-outputs/scorers-1.md)