> For the complete documentation index, see [llms.txt](https://docs.layerlens.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.layerlens.ai/4.1-general-use-cases/benchmark-driven-development.md).

# Benchmark-driven development

Make benchmarks a first-class signal in your dev loop — every change runs against a benchmark, every result is on the dashboard.

Most teams treat benchmarks as a one-time activity at the start of a project. Pick a model, glance at MMLU, get on with it. **Benchmark-driven development** flips that — every meaningful change to your AI feature runs against the benchmarks that matter, and every result lives on a shared dashboard.

## The shape of the work

1. **Pick your benchmarks.** Public benchmarks for general capability + your own private benchmark suite for task-specific quality.
2. **Wire it into your dev loop.** Local: `pytest`-style harness running a small slice. CI: full benchmark suite on every PR. Nightly: full suite + production-traffic slice.
3. **Track the score over time.** Stratix's evaluation history page shows the score curve per benchmark.
4. **Block regressions in CI.** A score that drops below baseline blocks the PR.
5. **Report wins to the team.** When a change improves a benchmark, share it.

## Why it works on Stratix

* **52+ public benchmarks ready to use** — you don't have to host the data
* **Private benchmark suites** — upload your own dataset and grading config
* **Score history** — every run is recorded; trends are visible
* **CI gates** — easily wired via the SDK or CLI
* **Compare models** — when a benchmark gets stronger, see if a different model is now a better fit

## Tools you'll use

* [Stratix Public — Benchmarks catalog](/5.-select-pick-the-model/benchmarks-catalog.md)
* [Stratix Premium — Benchmarks](https://github.com/LayerLens/gitbook-full/blob/main/13-reference/cli/benchmarks.md)
* [SDK: `client.evaluations.create()`](/4.1-general-use-cases/general.md)
* [CLI: `layerlens evaluate`](/4.1-general-use-cases/general.md)

## Outcomes you should see

You'll know this is working when:

* **Every PR touching AI runs a benchmark gate** — non-negotiable, not opt-in.
* **Your benchmark gating eval runs in <5 minutes** — fast enough that no one wants to skip it.
* **Score-over-time trends are visible to the whole team**, not buried in one engineer's terminal.
* **Net-new benchmarks reach 80% of your team within a week** — adding signal is cheap and shared.

## Anti-patterns

* **Benchmark inflation.** Adding 30 benchmarks because "more signal is better." Pick 3-5 that matter, run them often.
* **Treating public scores as your scores.** Public scores are a leading indicator. Your private benchmark on your data is the verdict.
* **Skipping CI.** If benchmarks don't gate merges, they're decoration.

## Where to next

* [Tutorial: Wire CI/CD quality gates](/6.-build-wire-your-code/03-cicd-gates.md)
* [Workflow: Evaluate](/9.-improve-tune-the-system/workflow.md)
* [Concept: Models and benchmarks](/5.-select-pick-the-model/models-and-benchmarks.md)
* [Cookbook: CI/CD recipes](/2.-get-started/all-cookbook-recipes.md)