Trace evaluations
Score uploaded traces with scorers and judges — recurring, on-demand, agentic.
Last updated
Was this helpful?
⌘Ctrlk
Score uploaded traces with scorers and judges — recurring, on-demand, agentic.
Available in Stratix Premium. This surface is part of the logged-in workspace at stratix.layerlens.ai. Stratix Public users can browse the catalog but cannot use this feature.
A trace evaluation runs a scoring config (scorers + judges) over a trace set. Use trace evaluations for:
Pre- and post-deployment agentic evaluation — score a curated trace set before shipping
Continuous evaluation — score live production traces on a recurring schedule
Ad-hoc inspection — grade a recent batch you just ingested
Create a trace evaluation against a trace set
Apply scorers and judges (any number, any combination)
Run on demand or on a recurring schedule
Browse past runs
Compare runs to a baseline
Get notified when scores cross thresholds
Pick the trace set — a saved set, a filter, or a single trace
Pick the scoring config — scorers + judges
Pick the schedule — one-off, daily, hourly
Configure thresholds (optional) — alert when a dimension drops below X
Run / save
Each trace evaluation result page shows:
Top-line score per dimension
Per-trace verdicts
Failed traces with span-level root-cause links
Score-over-time chart (for recurring runs)
Comparison to baseline (most recent prior successful run)
Pre-deploy: trace set is a curated test set. Run on a candidate change. Compare to last successful run.
Continuous: trace set is your production stream. Run on a schedule. Watch the trend.
For pre- and post-deployment agentic evaluations, the trace evaluation is the runtime. The criteria mix natural-language assertions, deterministic rules, and judges into one config — see Concept: Agentic evaluation.
Last updated
Was this helpful?
Was this helpful?