Glossary

Definitions for every Stratix term — models, evaluations, judges, scorers, traces, and more.

Agentic evaluation

Pre- and post-deployment evaluation of multi-step AI agents. Combines natural-language assertions, deterministic rules, and LLM judges to verify that an agent reaches a correct state, takes legal actions along the way, and satisfies regression criteria. Surfaced in Stratix Premium under Agent Evaluation and exercised at stratix.layerlens.ai/dashboard/agent-evaluation. See Agentic evaluations.

Benchmark

A standardized dataset and grading procedure used to measure model performance. Stratix hosts 52+ public benchmarks (MMLU, HumanEval, GSM8K, etc.) in the public catalog. See Benchmarks catalog.

BYOK (Bring Your Own Key)

The ability to use your own LLM provider API keys with Stratix. Two scopes:

• BYOK custom models — register an OpenAI-compatible endpoint as a custom model the Premium UI can target.
• BYOK provider keys — supply your own OpenAI/Anthropic/Google keys for evaluations that consume tokens.

See BYOK custom models and BYOK models concept.

Compare models

A Stratix Public and Premium feature that puts two models head-to-head across a benchmark or your own dataset. See Compare models.

Continuous evaluation

The practice of running evaluations on a recurring schedule against live traffic — not only at release time. Stratix supports continuous evaluation via trace-evaluations on real production traces. See Continuous evaluation.

ECU (Evaluation Compute Unit)

The Stratix billing unit for compute-intensive operations: model inference, judge runs, GEPA optimization, etc. Free tier includes a starter ECU balance; Premium tier is pay-as-you-go. See ECU credits.

Evaluation

A run that scores model output against a benchmark or dataset. Results are stored, browsable, and shareable. Two flavors:

• Public evaluations — runs visible to everyone on stratix.layerlens.ai.
• Private evaluations — runs scoped to your organization in stratix.layerlens.ai.

See Evaluations.

Evaluation space

A workspace bundling a model selection, dataset/benchmark selection, and scoring config. Spaces let you re-run the same evaluation as conditions change. Public spaces are shareable; Premium spaces are org-scoped. See Evaluation spaces.

GEPA judge optimization

Automatic tuning of an LLM judge's prompt against a labeled ground-truth set. GEPA finds the prompt that best matches your labels, raising judge agreement with humans. See Judge optimization (GEPA).

Judge

An LLM-based grader that scores model output along subjective dimensions (helpfulness, accuracy, faithfulness, etc.). Stratix Premium supports building custom judges, optimizing them with GEPA, and applying them to evaluations and traces. See Judges.

Model

An LLM (e.g., GPT-5.3, Claude Opus 4.6, Gemini 3.1 Pro). Stratix catalogs 175+ models with metadata, benchmark scores, and per-context information. See Models and benchmarks.

Multi-org / Organization

Stratix supports users belonging to multiple organizations. After sign-in, the user picks an active organization. Org-switch updates the session and reloads org-scoped data. See Organizations.

Public catalog

The set of models, benchmarks, evaluations, and spaces visible without authentication on stratix.layerlens.ai. See Stratix Public.

Quarterly Reports

Long-form research reports (Q1-Q4) summarizing model performance, benchmark winners, and AI evaluation trends. See Quarterly reports.

Scorer

A code grader (regex match, JSON-schema validation, exact match, semantic similarity, etc.). Scorers are fast and cheap and suit objective dimensions. See Scorers.

Span

A child unit of work within a trace. A span represents one tool call, one LLM call, or one logical step. See Traces and spans.

Stratix Premium

The logged-in workspace at stratix.layerlens.ai. Run private evaluations, build judges, score traces, manage scorers, run agentic evals, govern AI quality.

Stratix Public

The anonymous browsing experience at stratix.layerlens.ai. Browse models, benchmarks, public evaluations, public spaces, and compare models.

Trace

A record of an AI call (or a chain of calls) — inputs, outputs, latencies, costs, errors, and any spans. Traces are uploaded to Stratix Premium for inspection and trace-evaluation. See Traces and spans.

Trace evaluation

A scoring run applied to ingested traces using one or more scorers and judges. Lets you grade real production traffic. See Trace evaluations.

---

Seeded from the LayerLens "25 AI evaluation terms" blog post and the live product surface. If a term is missing, open an issue or PR.