Evaluation anatomy

The anatomy of a Stratix evaluation — every field, every state, every artifact.

A useful reference for what an evaluation actually contains, end-to-end.

At creation

Field	Required	Description
name	yes	Human-readable name
model_id or model	yes (if not a comparison)	The model under test
benchmark_id or dataset_id	yes	The data
scorers	optional	List of scorer IDs
judges	optional	List of judge IDs
tags	optional	For filtering and audit
parent_run_id	optional	Link to the previous baseline run

During execution

States: queued → running → completed (or failed, cancelled).

Per-row work:

1. Construct the prompt from the dataset row
2. Call the model
3. Apply each scorer to the (input, output, expected)
4. Apply each judge to the (input, output)
5. Record the row's verdicts, latency, cost

At completion

Field	Description
id	Stable identifier
score	Top-line score (depending on aggregation)
pass_rate	Pass rate over rows
judge_results[judge_id]	Per-judge aggregated verdict
scorer_results[scorer_id]	Per-scorer aggregated verdict
rows	Per-row details
latency_ms_p50/p95/p99	Latency distribution
cost_total	ECU consumed
created_at, completed_at	Timestamps

Artifacts

Each evaluation produces:

• The result object above
• A row-level results dataset (browsable, exportable)
• Optional: a comparison artifact if part of a compare-models run
• Optional: a regression artifact if there's a baseline

See also

• Evaluations
• Stratix Premium — Evaluations
• SDK: evaluations