> For the complete documentation index, see [llms.txt](https://docs.layerlens.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.layerlens.ai/more-in-this-section-7/cost-control-levers.md).

# Cost-control levers

ECU consumption follows from the workload. Below are the levers that move the bill without compromising signal.

## 1. BYOK custom models

When a [BYOK custom model](/5.-select-pick-the-model/byok-custom-models.md) is selected, model-inference tokens bill against **your** provider account, not Stratix ECU. Stratix ECU only bills the platform engine work (judges, scorers, GEPA, trace eval orchestration). For high-volume workloads, BYOK is the largest single lever.

## 2. Prompt caching pass-through

When the judge model supports prompt caching (Anthropic, OpenAI), Stratix passes through caching annotations on the rubric portion of the prompt. Repeated judge calls against the same rubric reuse the cached prefix.

**Effect:** judge calls beyond the first (per cache window) drop \~70-90% in token cost. The savings show up as lower ECU consumption automatically; no config required.

**Verification:** check the per-call cost in the trace evaluation results — the second-and-onward calls should be markedly cheaper.

## 3. Batch judges

For trace evaluations spanning many traces, Stratix batches judge calls when the underlying provider supports it (OpenAI Batch API, Anthropic Batch API where available).

**Effect:** \~50% cost reduction on batch-eligible workloads, at the cost of higher latency (batches resolve over hours rather than seconds).

**Use:** continuous-evaluation cadences — overnight scoring of the prior 24h of traces.

## 4. Sample, don't enumerate

For high-volume production traces, sampling is the right shape:

```python
trace_eval = client.trace_evaluations.create(
 trace_set={"sample_rate": 0.05, "tags": {"env": "production"}},
 judges=[...]
)
```

5% sampling gives statistically meaningful continuous-eval signal at 5% of the cost. Stratify the sample (by feature, by user tier, by tag) to avoid bias.

## 5. Anchor with deterministic primitives

Deterministic scorers cost \~zero. Judges are 1000-10000× more expensive per call.

**Pattern:** use scorers and rules to filter; reserve judges for residual subjective bar.

Example: a faithfulness scorer extracts every claim and checks each against the source via regex/embedding. The judge only runs on the small subset where the scorer flagged ambiguity.

## 6. Smaller judging model when feasible

Most rubrics work on a balanced judging model. Reserve the strongest (most expensive) judging model for genuinely subtle dimensions. After GEPA-optimizing the rubric, often a smaller model meets the same agreement rate.

## 7. Cache by content hash

For idempotent evaluations (same inputs, same outputs), Stratix caches results by content hash. Re-running an eval against an unchanged dataset is free after the first run.

**Use:** CI gates that run the same eval on every PR — the unchanged dataset rows aren't re-scored.

## 8. Cost preview before runs

The Premium dashboard shows worst-case ECU consumption before any compute-intensive run. Approve before it starts. Real consumption is usually lower.

## When the bill is too high

In order:

1. Check the Consumption tab in **Settings → Billing** — what's burning ECU?
2. Switch the judging model down a tier on judges that don't need the strongest one
3. Reduce sampling rate on continuous evaluations
4. Move to BYOK custom models for the highest-volume model inferences
5. Negotiate Volume ECU on Enterprise tier if monthly run-rate justifies it

## See also

* [Concept: ECU credits](/11.-admin/concepts-ecu-credits.md)
* [Premium — ECU credits and billing](/11.-admin/ecu-credits-billing.md)
* [Cookbook: cost-aware evaluation](/more-in-this-section-6/cost-aware-evaluation.md)
* [Reduce eval cost](/more-in-this-section-7/reduce-eval-cost.md)


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.layerlens.ai/more-in-this-section-7/cost-control-levers.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.