> For the complete documentation index, see [llms.txt](https://docs.layerlens.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.layerlens.ai/more-in-this-section-2/eval-patterns-5.md).

# Insurance evaluation patterns

Insurance evaluation patterns — primitive ratio, span rules, GEPA labeling, cadence.

The recommended evaluation-primitive ratio for insurance AI is **65/25/10** — deterministic rules, code assertions, and judges respectively. Insurance has more rule-bound regulatory surface than most industries (state DOI rate filings, statute deadlines, sub-limit math) so rules and assertions dominate.

## Primitive ratio

| Primitive           | Share | Why                                                                                |
| ------------------- | ----- | ---------------------------------------------------------------------------------- |
| Deterministic rules | 65%   | State filings, document-version match, dollar-amount math, SoL deadlines           |
| Code assertions     | 25%   | Policy-section citation existence, premium-band feasibility, SIU reason-code count |
| LLM judges          | 10%   | Explainability, hedge-language quality, liability-finding rationale                |

Reserve judges for "does this read like a regulator-defensible explanation" tasks. Use rules everywhere a number, date, or document version can be checked.

## Pattern table

| Scenario                                | Primary primitive | Specific check                                       |
| --------------------------------------- | ----------------- | ---------------------------------------------------- |
| Claims triage — severity bucket         | Rule              | Severity ≥ "major" auto-routes to senior adjuster    |
| Claims triage — coverage citation       | Code assertion    | Cited policy section ID exists in policy file        |
| Claims triage — photo presence          | Rule              | Required-photo flag set when policy demands it       |
| Underwriting — fairness                 | Code scorer       | Disparate impact ratio ≥ 0.8 (four-fifths)           |
| Underwriting — premium justification    | Rule              | Every premium decision cites a rate filing factor    |
| Underwriting — state rule applicability | Rule              | State-specific exclusions match policyholder state   |
| Underwriting — explainability           | Judge             | Reasoning is regulator-followable                    |
| Policy Q\&A — document grounding        | Judge             | Faithfulness to customer's policy version            |
| Policy Q\&A — version match             | Rule              | Cited doc version = active version                   |
| Policy Q\&A — sub-limit math            | Rule              | Dollar amounts in output match policy file           |
| Policy Q\&A — hedge language            | Rule              | Required disclaimer string present                   |
| Fraud — top-decile precision            | Code assertion    | High-score band matches confirmed-fraud rate         |
| Fraud — referral fairness               | Code scorer       | Group-rate ratio within tolerance                    |
| Subrogation — SoL deadline              | Rule              | Every subrogation file has SoL date and jurisdiction |

## Span rules

Insurance traces should expose at minimum these spans:

* `policy.lookup` — which policy version was retrieved
* `coverage.eval` — which exclusions and endorsements applied
* `rate.filing.cite` — which filed factor justified the premium
* `siu.scoring` — fraud score and contributing factors
* `subrogation.identify` — liable-party identification and SoL computation

Span-level scorers run on each span independently. Failure of any one fails the trace.

## GEPA labeling

For each insurance judge, label ≥ 50 paired examples drawn from real (de-identified) claim files:

* **Claims triage:** 30 routine + 15 escalations + 5 fraud-flagged
* **Underwriting:** 25 standard + 15 substandard + 10 declined
* **Policy Q\&A:** stratified across coverage types and policy form versions
* **Fraud SIU:** match labeled-fraud set with non-fraud controls (1:3 ratio is typical)
* **Subrogation:** labeled by senior subrogation attorney

Run [judge optimization](/more-in-this-section-9/judge-optimizations-1.md) with `budget="medium"` until agreement-with-humans ≥ 92%. Drop iteration if the optimized judge regresses on a held-out 20% slice.

## Cadence

| Cadence       | Activity                                                                      |
| ------------- | ----------------------------------------------------------------------------- |
| **Per-PR**    | Run the scenario test set as a CI gate; deterministic rules + assertions only |
| **Daily**     | Sample 1-5% of production traces; run all judges + rules                      |
| **Weekly**    | Drift report on score distributions; underwriting fairness audit              |
| **Monthly**   | GEPA re-optimization of judges if drift detected                              |
| **Quarterly** | Bench against a frozen scenario suite to catch silent regressions             |
| **Annually**  | DOI examination prep — pull last 12 months of evaluations and rule outcomes   |

## See also

* [Insurance scenarios](/more-in-this-section-2/scenarios-5.md)
* [Insurance compliance](/more-in-this-section-2/compliance-5.md)
* [Recipe: GEPA basics](/9.-improve-tune-the-system/gepa-basics.md)
* [Trace evaluations (SDK)](/more-in-this-section-9/trace-evaluations-1.md)