> For the complete documentation index, see [llms.txt](https://docs.layerlens.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.layerlens.ai/7.-observe-see-whats-happening/continuous-evaluation.md).

# Continuous evaluation

Continuous evaluation — score live production traces on a recurring schedule.

Continuous evaluation scores live production traces on a recurring schedule. Pre- and post-deployment evaluations catch regressions before merge; continuous evaluation catches them after.

## Why it matters

* Production traffic shifts; what worked in pre-deploy may fail in the wild
* Model providers update their backends; your scores may move quietly
* Prompt or feature changes that slipped through CI surface here

Without continuous evaluation, AI quality is a post-incident discovery. With it, regressions surface before customer-facing impact.

## The shape

1. **Ingest live traces** — from your application, via SDK or API
2. **Define a trace evaluation** — scorers + judges
3. **Schedule it** — daily, hourly, per-batch
4. **Watch the trend** — score-over-time per dimension
5. **Get notified on drift** — thresholds, alerts

## Pre-deploy vs continuous patterns

| Aspect       | Pre-deploy agentic  | Continuous                  |
| ------------ | ------------------- | --------------------------- |
| Trace set    | curated, fixed      | rolling, sampled production |
| Cadence      | per-PR, per-release | daily, hourly               |
| Bar          | block merge         | alert on drift              |
| Cost profile | bursty              | steady                      |

## Sampling

For high-volume traffic, sample. Sample bias is the trap — production distribution drifts, and sampling drifts with it. Re-validate the sampling strategy periodically.

## Cost management

Continuous evaluation can become a big consumer of ECU. Two levers:

* **Sample less of low-importance traffic, more of high-stakes**
* **Use scorers liberally; reserve judges for residual subjective bar**

## Where to next

* [Trace evaluations (Premium)](/8.-evaluate-score-the-outputs/trace-evaluations.md)
* [Stratix Premium — Notifications](https://github.com/LayerLens/gitbook-full/blob/main/13-reference/sdk-python/notifications.md)
* [Use case: Continuous evaluation](/7.-observe-see-whats-happening/continuous-evaluation.md)
* [Tutorial: Score live traces](/8.-evaluate-score-the-outputs/04-score-traces.md)