# Pattern: customer-service pricing accuracy

A telecom AI customer-service agent handles billing questions, plan changes, and pricing inquiries. The agent reads the active rate card and applies promotional pricing rules. When it gets the math right, churn drops and customer-care hours fall. When it gets it wrong — even occasionally — the bill is wrong and revenue leaks.

This pattern shows how to evaluate pricing accuracy continuously, in CI and in production.

## What's at stake

| Risk dimension                                             | Magnitude                         | Framework                          |
| ---------------------------------------------------------- | --------------------------------- | ---------------------------------- |
| Annual revenue leakage from underbilling at 6% error rate  | \~$2M for a mid-market carrier    | Internal billing-reconciliation    |
| FCC complaint exposure from overcharging                   | Per-violation civil-penalty bands | FCC enforcement bureau guidance    |
| Per-subscriber churn cost from a billing-dispute departure | $300–$600                         | Telecom retention benchmarks       |
| Audit / SOX exposure on unreviewed pricing-engine outputs  | Material-weakness risk            | Public-company controls frameworks |

## The evaluation pattern

**Pre- and post-deployment, in CI:** a fast (≤5 min) regression evaluation runs on every PR touching the customer-service prompt or pricing logic.

1. **JSON-schema scorer** — every pricing change in the agent's output must include a `rate_card_row_id` and a citation timestamp. Missing field = fail.
2. **Numeric scorer** — extracted prices match the rate-card row exactly (no rounding, no unit substitution).
3. **Compare-models** view — when proposing a prompt variant, compare the variant against the current main-branch baseline on the same regression set. Pick the variant with the lowest error rate.
4. **Tolerance gate** — fail the build if pricing-error rate exceeds 0.5%.

**Post-deployment, continuous:** trace evaluation on a 5% sample of live customer interactions, hourly cadence.

* Same JSON-schema and numeric scorers
* Threshold alert routed to a Slack channel below 99.5% pass rate
* Per-trace alert on any CRITICAL error (extracted price differs from rate card by >$5)

## Configuration in code

```python
# Python (SDK) — CI-side gating evaluation
from layerlens import Stratix
client = Stratix()

schema_scorer = client.scorers.create(
 name="pricing-citation",
 type="json_schema_validate",
 config={"schema": {
 "type": "object",
 "required": ["rate_card_row_id", "citation_ts"],
 "properties": {
 "rate_card_row_id": {"type": "string"},
 "citation_ts": {"type": "string", "format": "date-time"},
 },
 }},
)

numeric_scorer = client.scorers.create_code(
 name="price-exact-match",
 code="result = abs(output['price'] - expected['rate_card_price']) < 0.01",
)

evaluation = client.evaluations.create(
 name="cs-pricing-gate",
 model_id=os.environ["MODEL_ID"],
 dataset_id="pricing-regression-v1",
 scorers=[schema_scorer.id, numeric_scorer.id],
)
result = client.evaluations.wait_for_completion(evaluation.id)
sys.exit(0 if result.pass_rate >= 0.995 else 1)
```

```typescript
// TypeScript (REST) — same gate from a Node CI runner
const r = await fetch("https://stratix.layerlens.ai/api/v1/evaluations", {
 method: "POST",
 headers: {
 "X-API-Key": process.env.LAYERLENS_STRATIX_API_KEY!,
 "Content-Type": "application/json",
 },
 body: JSON.stringify({
 name: "cs-pricing-gate",
 model_id: process.env.MODEL_ID,
 dataset_id: "pricing-regression-v1",
 scorers: [schemaScorerId, numericScorerId],
 }),
});
const evaluation = await r.json();
// Poll evaluation.id until status === "completed"
```

## What you get

* Pricing regressions caught at PR — never reach production.
* Continuous evaluation surfaces drift when the rate card updates without the prompt being refreshed.
* Revenue reconciliation moves from discovery surface to verification surface.
* Per-call request IDs in Stratix evaluation results pair with internal call logs for fast incident triage.

## Stratix capabilities used

* [SDK + CLI evaluations](https://github.com/LayerLens/gitbook-full/blob/main/13-reference/sdk-python/evaluations.md) wired into [GitHub Actions](/6.-build-wire-your-code/integration-github-actions.md)
* [Compare models](/5.-select-pick-the-model/compare-models.md) — prompt-variant A/B
* [JSON-schema and numeric scorers](https://github.com/LayerLens/gitbook-full/blob/main/08-evaluate/cookbook/json-schema-validate.md)
* [Trace evaluations](/8.-evaluate-score-the-outputs/trace-evaluations.md) — continuous sampled
* [Notifications](https://github.com/LayerLens/gitbook-full/blob/main/13-reference/sdk-python/notifications.md) — Slack routing on threshold

## Replicate this

**Get started:** [Tutorial 3: Wire CI/CD quality gates](/6.-build-wire-your-code/03-cicd-gates.md) walks the end-to-end build (\~30 min).

* [Use case: AI quality gates in CI/CD](/4.1-general-use-cases/ai-quality-gates-cicd.md)
* [Industry → Telecommunications](/4.2-industry-use-cases/telecommunications.md)
* [Workflow: Govern](/9.-improve-tune-the-system/workflow.md)
* [Cookbook: cost-aware evaluation](https://github.com/LayerLens/gitbook-full/blob/main/08-evaluate/cookbook/cost-aware-evaluation.md)


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.layerlens.ai/4.2-industry-use-cases/pattern-10.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.