Your first evaluation
Run your first private evaluation in Stratix Premium — pick a model, pick a dataset, see scores in minutes.
Last updated
Was this helpful?
⌘Ctrlk
Run your first private evaluation in Stratix Premium — pick a model, pick a dataset, see scores in minutes.
A private evaluation runs a model against a dataset and scores the outputs. Here's the fastest way to do one.
In the left rail, click Evaluations. Click New evaluation.
Search and select. The full public catalog is searchable, plus any BYOK custom models you've registered.
Two options:
Upload a dataset — drag a CSV/JSONL onto the upload zone. Stratix infers the schema; you confirm input and expected-output columns.
Use a starter dataset — pick a benchmark from the public catalog (e.g., MMLU subset).
For your first eval, pick a code grader that fits your data:
Exact match — outputs equal to expected
Semantic similarity — fuzzy match using an embedding model
Regex match — outputs match a pattern
JSON schema — outputs validate against a schema
Skip judges for now — you'll build one later.
Stratix shows the worst-case ECU consumption before you run. Approve.
Click Run. The evaluation queues; results stream in row-by-row.
The results page shows:
Overall score
Per-row scores (input → output → score)
Score distribution
Latency and cost per row
You should have a completed evaluation with a top-line score. Click any failing row to see the input, the model output, and the expected output side-by-side.
Stratix called the model on each input row, ran the scorer against each output, and aggregated the result. Every row is stored — you can re-open this evaluation later or compare it against a future run.
Run the same evaluation against a different model. Stratix's compare-models view shows both runs side-by-side.
Iterate on the prompt. If your dataset includes a prompt template column, change the template and rerun.
Add a judge. See First judge for subjective dimensions.
Last updated
Was this helpful?
Was this helpful?