> For the complete documentation index, see [llms.txt](https://docs.layerlens.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.layerlens.ai/more-in-this-section-2/eval-patterns-11.md).

# Telecommunications evaluation patterns

Recommended primitive ratio for telecom AI: **65/25/10** — deterministic rules, code assertions, judges. Carrier policy, regulatory rules, and lawful-intercept scope are tightly bounded; judges focus on customer-facing copy quality.

## Primitive ratio

| Primitive           | Share | Why                                                                                                   |
| ------------------- | ----- | ----------------------------------------------------------------------------------------------------- |
| Deterministic rules | 65%   | Authentication, NORS triggers, advisory-only on network changes, scope-limitation on lawful intercept |
| Code assertions     | 25%   | Top-decile precision on fraud, fairness, freshness thresholds                                         |
| LLM judges          | 10%   | Plan-promise faithfulness, customer-impact statements, multilingual parity                            |

## Pattern table

| Scenario                                       | Primary primitive | Specific check                                       |
| ---------------------------------------------- | ----------------- | ---------------------------------------------------- |
| Customer service — plan promise                | Rule + Judge      | Promise matches policy                               |
| Customer service — outage freshness            | Rule              | Cited data within threshold                          |
| Customer service — authentication              | Rule              | Verified identity for changes                        |
| NOC — mitigation faithfulness                  | Judge             | Output grounded in runbook                           |
| NOC — NORS trigger                             | Rule              | 911-affecting outages flag NORS                      |
| NOC — advisory-only                            | Rule              | AI never auto-applies changes                        |
| Plan recommendations — promotion realizability | Rule              | Achievable as offered                                |
| Plan recommendations — negative-option guard   | Rule              | Affirmative consent required                         |
| Plan recommendations — fairness                | Code scorer       | Group-level offered-price parity                     |
| Fraud — top-decile precision                   | Code assertion    | Match confirmed fraud                                |
| Fraud — fairness                               | Code scorer       | Group-level lockout parity                           |
| Fraud — recovery path                          | Rule              | Lockout has documented recovery                      |
| Lawful intercept — order type                  | Rule              | Subpoena/2703(d)/warrant/Title III correctly classed |
| Lawful intercept — scope                       | Rule              | Response bounded by order                            |
| Lawful intercept — attorney gate               | Rule              | No production without sign-off                       |

## Span rules

* `account.lookup` — customer state, plan, identity-verification
* `policy.cite` — policy section reference
* `outage.lookup` — outage data freshness and source
* `network.advisory` — flag confirming advisory-only
* `fraud.score` — score and contributing signals
* `intercept.order` — order classification and scope

## GEPA labeling

For each telecom judge, label ≥ 50 paired examples:

* **Plan-promise faithfulness:** CX leads + policy team label
* **Customer-impact statements:** Comms team + legal label
* **Multilingual parity:** Native-speaker QA per language

Run [judge optimization](/more-in-this-section-9/judge-optimizations-1.md) with `budget="medium"`. Hold out 20%.

## Cadence

| Cadence        | Activity                                                |
| -------------- | ------------------------------------------------------- |
| **Per-deploy** | CI gate runs scenario suite (rules + assertions)        |
| **Continuous** | Outage-data freshness; NORS-trigger watch               |
| **Daily**      | Sample production traces; fraud P/R report              |
| **Weekly**     | Fairness audit; plan-promise audit; SIM-swap rate watch |
| **Monthly**    | GEPA re-optimization on judges                          |
| **Quarterly**  | FCC reporting prep; STIR/SHAKEN compliance audit        |
| **Annually**   | CALEA capability audit; full bias audit                 |

## See also

* [Telecom scenarios](/more-in-this-section-2/scenarios-11.md)
* [Telecom compliance](/more-in-this-section-2/compliance-11.md)
* [Recipe: GEPA basics](/9.-improve-tune-the-system/gepa-basics.md)


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.layerlens.ai/more-in-this-section-2/eval-patterns-11.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.