> For the complete documentation index, see [llms.txt](https://docs.layerlens.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.layerlens.ai/more-in-this-section-2/eval-patterns-11.md).

# Telecommunications evaluation patterns

Telecommunications evaluation patterns — primitive ratio, span rules, GEPA labeling, cadence.

Recommended primitive ratio for telecom AI: **65/25/10** — deterministic rules, code assertions, judges. Carrier policy, regulatory rules, and lawful-intercept scope are tightly bounded; judges focus on customer-facing copy quality.

## Primitive ratio

| Primitive           | Share | Why                                                                                                   |
| ------------------- | ----- | ----------------------------------------------------------------------------------------------------- |
| Deterministic rules | 65%   | Authentication, NORS triggers, advisory-only on network changes, scope-limitation on lawful intercept |
| Code assertions     | 25%   | Top-decile precision on fraud, fairness, freshness thresholds                                         |
| LLM judges          | 10%   | Plan-promise faithfulness, customer-impact statements, multilingual parity                            |

## Pattern table

| Scenario                                       | Primary primitive | Specific check                                       |
| ---------------------------------------------- | ----------------- | ---------------------------------------------------- |
| Customer service — plan promise                | Rule + Judge      | Promise matches policy                               |
| Customer service — outage freshness            | Rule              | Cited data within threshold                          |
| Customer service — authentication              | Rule              | Verified identity for changes                        |
| NOC — mitigation faithfulness                  | Judge             | Output grounded in runbook                           |
| NOC — NORS trigger                             | Rule              | 911-affecting outages flag NORS                      |
| NOC — advisory-only                            | Rule              | AI never auto-applies changes                        |
| Plan recommendations — promotion realizability | Rule              | Achievable as offered                                |
| Plan recommendations — negative-option guard   | Rule              | Affirmative consent required                         |
| Plan recommendations — fairness                | Code scorer       | Group-level offered-price parity                     |
| Fraud — top-decile precision                   | Code assertion    | Match confirmed fraud                                |
| Fraud — fairness                               | Code scorer       | Group-level lockout parity                           |
| Fraud — recovery path                          | Rule              | Lockout has documented recovery                      |
| Lawful intercept — order type                  | Rule              | Subpoena/2703(d)/warrant/Title III correctly classed |
| Lawful intercept — scope                       | Rule              | Response bounded by order                            |
| Lawful intercept — attorney gate               | Rule              | No production without sign-off                       |

## Span rules

* `account.lookup` — customer state, plan, identity-verification
* `policy.cite` — policy section reference
* `outage.lookup` — outage data freshness and source
* `network.advisory` — flag confirming advisory-only
* `fraud.score` — score and contributing signals
* `intercept.order` — order classification and scope

## GEPA labeling

For each telecom judge, label ≥ 50 paired examples:

* **Plan-promise faithfulness:** CX leads + policy team label
* **Customer-impact statements:** Comms team + legal label
* **Multilingual parity:** Native-speaker QA per language

Run [judge optimization](/more-in-this-section-9/judge-optimizations-1.md) with `budget="medium"`. Hold out 20%.

## Cadence

| Cadence        | Activity                                                |
| -------------- | ------------------------------------------------------- |
| **Per-deploy** | CI gate runs scenario suite (rules + assertions)        |
| **Continuous** | Outage-data freshness; NORS-trigger watch               |
| **Daily**      | Sample production traces; fraud P/R report              |
| **Weekly**     | Fairness audit; plan-promise audit; SIM-swap rate watch |
| **Monthly**    | GEPA re-optimization on judges                          |
| **Quarterly**  | FCC reporting prep; STIR/SHAKEN compliance audit        |
| **Annually**   | CALEA capability audit; full bias audit                 |

## See also

* [Telecom scenarios](/more-in-this-section-2/scenarios-11.md)
* [Telecom compliance](/more-in-this-section-2/compliance-11.md)
* [Recipe: GEPA basics](/9.-improve-tune-the-system/gepa-basics.md)