> For the complete documentation index, see [llms.txt](https://docs.layerlens.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.layerlens.ai/more-in-this-section-2/eval-patterns-5.md). # Insurance evaluation patterns The recommended evaluation-primitive ratio for insurance AI is **65/25/10** — deterministic rules, code assertions, and judges respectively. Insurance has more rule-bound regulatory surface than most industries (state DOI rate filings, statute deadlines, sub-limit math) so rules and assertions dominate. ## Primitive ratio | Primitive | Share | Why | | ------------------- | ----- | ---------------------------------------------------------------------------------- | | Deterministic rules | 65% | State filings, document-version match, dollar-amount math, SoL deadlines | | Code assertions | 25% | Policy-section citation existence, premium-band feasibility, SIU reason-code count | | LLM judges | 10% | Explainability, hedge-language quality, liability-finding rationale | Reserve judges for "does this read like a regulator-defensible explanation" tasks. Use rules everywhere a number, date, or document version can be checked. ## Pattern table | Scenario | Primary primitive | Specific check | | --------------------------------------- | ----------------- | ---------------------------------------------------- | | Claims triage — severity bucket | Rule | Severity ≥ "major" auto-routes to senior adjuster | | Claims triage — coverage citation | Code assertion | Cited policy section ID exists in policy file | | Claims triage — photo presence | Rule | Required-photo flag set when policy demands it | | Underwriting — fairness | Code scorer | Disparate impact ratio ≥ 0.8 (four-fifths) | | Underwriting — premium justification | Rule | Every premium decision cites a rate filing factor | | Underwriting — state rule applicability | Rule | State-specific exclusions match policyholder state | | Underwriting — explainability | Judge | Reasoning is regulator-followable | | Policy Q\&A — document grounding | Judge | Faithfulness to customer's policy version | | Policy Q\&A — version match | Rule | Cited doc version = active version | | Policy Q\&A — sub-limit math | Rule | Dollar amounts in output match policy file | | Policy Q\&A — hedge language | Rule | Required disclaimer string present | | Fraud — top-decile precision | Code assertion | High-score band matches confirmed-fraud rate | | Fraud — referral fairness | Code scorer | Group-rate ratio within tolerance | | Subrogation — SoL deadline | Rule | Every subrogation file has SoL date and jurisdiction | ## Span rules Insurance traces should expose at minimum these spans: * `policy.lookup` — which policy version was retrieved * `coverage.eval` — which exclusions and endorsements applied * `rate.filing.cite` — which filed factor justified the premium * `siu.scoring` — fraud score and contributing factors * `subrogation.identify` — liable-party identification and SoL computation Span-level scorers run on each span independently. Failure of any one fails the trace. ## GEPA labeling For each insurance judge, label ≥ 50 paired examples drawn from real (de-identified) claim files: * **Claims triage:** 30 routine + 15 escalations + 5 fraud-flagged * **Underwriting:** 25 standard + 15 substandard + 10 declined * **Policy Q\&A:** stratified across coverage types and policy form versions * **Fraud SIU:** match labeled-fraud set with non-fraud controls (1:3 ratio is typical) * **Subrogation:** labeled by senior subrogation attorney Run [judge optimization](/more-in-this-section-9/judge-optimizations-1.md) with `budget="medium"` until agreement-with-humans ≥ 92%. Drop iteration if the optimized judge regresses on a held-out 20% slice. ## Cadence | Cadence | Activity | | ------------- | ----------------------------------------------------------------------------- | | **Per-PR** | Run the scenario test set as a CI gate; deterministic rules + assertions only | | **Daily** | Sample 1-5% of production traces; run all judges + rules | | **Weekly** | Drift report on score distributions; underwriting fairness audit | | **Monthly** | GEPA re-optimization of judges if drift detected | | **Quarterly** | Bench against a frozen scenario suite to catch silent regressions | | **Annually** | DOI examination prep — pull last 12 months of evaluations and rule outcomes | ## See also * [Insurance scenarios](/more-in-this-section-2/scenarios-5.md) * [Insurance compliance](/more-in-this-section-2/compliance-5.md) * [Recipe: GEPA basics](/9.-improve-tune-the-system/gepa-basics.md) * [Trace evaluations (SDK)](/more-in-this-section-9/trace-evaluations-1.md) --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://docs.layerlens.ai/more-in-this-section-2/eval-patterns-5.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.