> For the complete documentation index, see [llms.txt](https://docs.layerlens.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.layerlens.ai/more-in-this-section-1/first-eval-5-minutes.md).

# Run your first evaluation in 5 minutes

One Python file. One model. One benchmark. One score. That's it.

This page is the fastest possible path from zero to a real evaluation. If something breaks, jump straight to [Common first-day errors](/2.-get-started/first-day-errors.md) and come back.

{% hint style="info" %}
**Prereqs:** Python 3.9+, a Stratix Premium account, and an API key from **Settings → API keys**. You'll need \~10 ECU of starter credit (every new org gets some).
{% endhint %}

## Step 1 — Install and authenticate (60 seconds)

```bash
pip install layerlens --extra-index-url https://sdk.layerlens.ai/package
export LAYERLENS_STRATIX_API_KEY="sk-..." # macOS / Linux
# PowerShell: $env:LAYERLENS_STRATIX_API_KEY = "sk-..."
```

Verify the install:

```bash
python -c "import layerlens; print(layerlens.__version__)"
```

You should see `1.3.0` or later. If not, see [first-day errors](/2.-get-started/first-day-errors.md).

## Step 2 — The whole thing in one file

Save this as `first_eval.py` and run it. That's the entire 5-minute experience.

```python
"""
first_eval.py — your first end-to-end Stratix evaluation.

Run with: python first_eval.py
Requires: LAYERLENS_STRATIX_API_KEY set in your environment.
"""
from layerlens import Stratix

# 1) Authenticate. The Stratix() constructor reads LAYERLENS_STRATIX_API_KEY
# from the environment. Pass api_key="..." explicitly if you prefer.
client = Stratix()

# 2) Pick a model from the public catalog. get_by_key() is case-sensitive
# and uses the "<provider>/<model>" key shown in the Catalog UI.
model = client.models.get_by_key("anthropic/claude-opus-4-7")
print(f"Model: {model.name}")

# 3) Pick a benchmark. MMLU is a fast, well-known public benchmark — good
# for a smoke test on day one.
benchmark = client.benchmarks.get_by_key("mmlu")
print(f"Benchmark: {benchmark.name}")

# 4) Create the evaluation. The platform queues it; the SDK returns
# immediately with an Evaluation object you can poll.
evaluation = client.evaluations.create(
 model=model,
 benchmark=benchmark,
)
print(f"Started: {evaluation.id} (status={evaluation.status})")

# 5) Block until the evaluation finishes. wait_for_completion polls every
# interval_seconds and gives up after timeout_seconds. MMLU on Opus 4.7
# typically completes in 2–4 minutes.
evaluation = client.evaluations.wait_for_completion(
 evaluation,
 interval_seconds=10,
 timeout_seconds=600,
)

if not evaluation.is_success:
 raise SystemExit(f"Evaluation did not succeed: status={evaluation.status}")

print(f"Done: accuracy={evaluation.accuracy:.3f}")

# 6) Pull per-prompt results. get_all_by_id auto-paginates across pages and
# returns a flat list of Result objects (.prompt,.result,.truth,.score).
results = client.results.get_all_by_id(evaluation.id)
print(f"Rows: {len(results)}")

# 7) Show the first three rows so you can see what real data looks like.
for r in results[:3]:
 verdict = "PASS" if r.score and r.score >= 0.5 else "FAIL"
 print(f" [{verdict}] {r.prompt[:60]}... score={r.score}")

# 8) Done. The same evaluation now appears in Premium → Evaluations.
print(f"\nView in UI: https://stratix.layerlens.ai/evaluations/{evaluation.id}")
```

## Step 3 — Run it

```bash
python first_eval.py
```

You should see something like:

```
Model: Claude Opus 4.7
Benchmark: MMLU
Started: eval_01HZ... (status=queued)
Done: accuracy=0.873
Rows: 14042
 [PASS] Which of the following best describes the function of... score=1.0
 [PASS] In the context of macroeconomic policy, the term 'cro... score=1.0
 [FAIL] A 65-year-old patient presents with acute onset chest... score=0.0

View in UI: https://stratix.layerlens.ai/evaluations/eval_01HZ...
```

Open that URL. The dashboard shows the same accuracy number, plus a per-subset breakdown, error analysis, and ECU consumed.

## You just did five things

1. **Selected** a model and a benchmark from the public catalog.
2. **Built** the evaluation by passing both objects to `evaluations.create`.
3. **Observed** the run via `wait_for_completion` (queued → running → completed).
4. **Evaluated** the model against MMLU's ground-truth labels — an LLM-backed scorer compares each model response to truth and emits a 0/1 score per row.
5. **Improved** your understanding of where the model wins and loses, by inspecting per-row results.

Those five stages — Select, Build, Observe, Evaluate, Improve — are the spine of every Stratix workflow.

## Common failure modes

* **`AuthenticationError: invalid API key`** — env var name is `LAYERLENS_STRATIX_API_KEY`, not `LAYERLENS_API_KEY`. See [first-day errors](/2.-get-started/first-day-errors.md).
* **`NotFoundError` on `get_by_key`** — keys are case-sensitive. Browse the catalog at `https://stratix.layerlens.ai/catalog` to copy the exact key.
* **Timeout** — bump `timeout_seconds`. Some benchmarks (HumanEval+, GPQA) take longer than 10 minutes on slower models.
* **"No ECU"** — every new org gets starter ECU; if you've exhausted it, top up under **Admin → Billing**.

## Where to next

You ran one model on one benchmark. Logical next steps:

* **Compare two models.** Repeat Steps 2–5 with a different model key and diff the accuracy. [Public — compare two models](/2.-get-started/compare-two-models.md) walks the UI version of this flow.
* **Score subjective dimensions.** Accuracy works when there's a ground-truth answer. For helpfulness, faithfulness, tone, etc., you need an LLM judge. [Tutorial 2: First judge](/8.-evaluate-score-the-outputs/02-first-judge.md).
* **Run it on your own data.** Swap the benchmark for an uploaded dataset. [Upload a private benchmark](/more-in-this-section-3/upload-benchmark.md).
* **Wire this into CI.** Promote your `first_eval.py` into a quality gate. [Tutorial 3: CI/CD gates](/6.-build-wire-your-code/03-cicd-gates.md).
* **Score live traces.** Score traffic that's already happening in production. [Tutorial 4: Score live traces](/8.-evaluate-score-the-outputs/04-score-traces.md).

If you'd rather follow a guided five-tutorial path, start at [All tutorials](/2.-get-started/all-tutorials.md). If you want to lift-and-shift a working snippet for a specific problem, jump to [Top 5 starter recipes](/more-in-this-section-1/top-5-starter-recipes.md).


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.layerlens.ai/more-in-this-section-1/first-eval-5-minutes.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.