Pattern: claims-triage agent

Insurance — claims-triage agent pattern. Span-level escalation rule, assertions, GEPA-tuned tone judge, in one agentic evaluation.

A property-and-casualty claims-triage agent extracts loss details from photos and submitted forms, classifies severity, estimates repair cost, and decides whether to auto-approve, request more information, or escalate to a human adjuster. Multi-step, tool-using, customer-facing on the empathy side.

This pattern shows how to evaluate the agent end-to-end with one agentic evaluation that mixes hard policy rules with subjective tone scoring.

What's at stake

Risk dimension	Magnitude	Framework
Annual industry insurance fraud (US, all lines)	~$80B	Coalition Against Insurance Fraud
Per-catastrophe-event mis-estimation cost	$50K–$500K	Industry loss-adjustment benchmarks
State DOI market-conduct examination	Per-finding penalty bands	NAIC market-conduct framework
Class-action exposure on systemic underpayment	Multi-million-dollar settlements	Public class-action filings

The evaluation pattern

Pre- and post-deployment agentic evaluation over a 400-trace curated set spanning happy paths, ambiguous documentation, and known-edge fraud patterns.

Criteria mix:

1. Deterministic span-level rule — "if inputs.claim_amount > $X then the trace must contain a span with kind=tool, tool=escalate_to_adjuster". CRITICAL severity on miss. The amount threshold is configurable per line of business.
2. Natural-language assertions — the agent's final state matches the policy auto-approval rules; every required document is present in the agent's verification span.
3. Faithfulness judge (GEPA-tuned against ≥50 examples — scored output) — claims about coverage and limits are grounded in the policy document.
4. Tone judge (GEPA-tuned against ≥50 examples — scored output) — for customer-facing outputs, balance professionalism with appropriate empathy in disruption scenarios.

Don't have labels yet? See Bootstrap a judge before GEPA for the week-1 setup.

Verdict + root-cause artifact: every failed criterion ties back to the specific trace, span, and decision that broke. The regression report compares the run to the most recent baseline; newly-failing criteria surface immediately.

Post-deployment, continuous: trace evaluation on a 5% sample of production claims, daily cadence. Same criteria configuration. Threshold alerts route to the claims-ops Slack channel.

Configuration in code

# Python (SDK) — agentic evaluation with span-level escalation rule
from layerlens import Stratix
client = Stratix()

escalation_rule = {
 "id": "high-value-must-escalate",
 "expression": "if input.claim_amount > 100000 then any(span.kind == 'tool' and span.tool == 'escalate_to_adjuster' for span in trace.spans)",
 "severity": "CRITICAL",
}

faithfulness = client.judges.create(
 name="policy-faithfulness",
 evaluation_goal="Coverage and limit claims must be grounded in the cited policy document.",
)

tone = client.judges.create(
 name="empathy-tone",
 evaluation_goal="In disruption scenarios, the response balances professionalism with appropriate empathy. Penalize tone-deaf or dismissive language.",
)

trace_eval = client.trace_evaluations.create(
 trace_set={"id": "claims-curated-400"},
 rules=[escalation_rule],
 judges=[faithfulness.id, tone.id],
)
result = client.trace_evaluations.wait_for_completion(trace_eval.id)
assert result.critical_failures == 0, "Block release: escalation rule violated"

// TypeScript (REST)
const r = await fetch("https://stratix.layerlens.ai/api/v1/trace-evaluations", {
 method: "POST",
 headers: {
 "X-API-Key": process.env.LAYERLENS_STRATIX_API_KEY!,
 "Content-Type": "application/json",
 },
 body: JSON.stringify({
 trace_set: { id: "claims-curated-400" },
 rules: [escalationRule],
 judges: [faithfulnessId, toneId],
 }),
});
const traceEval = await r.json();

What you get

• Zero CRITICAL escalation-rule violations across the curated trace set before launch.
• Continuous trace evaluation catches drift when policy rules or product mix shifts.
• Verdict + root-cause artifacts become the auditor-ready evidence packet for state DOI inquiries — every release pin cites the evaluation IDs that gated it.
• Tone-judge agreement with human reviewers above 90% after GEPA optimization, calibrated to your team's customer-experience standard.

Stratix capabilities used

• Agentic evaluation — three criteria types in one configuration
• Span-level scoring — escalation rule pinned to tool spans
• Judges with GEPA optimization — faithfulness and tone
• Trace evaluations — pre-deploy and continuous
• Notifications — Slack routing
• Audit-friendly evidence

Replicate this

• Industry → Insurance
• Cookbook: detect tool-call regressions in agents
• Cookbook: insurance claims-triage scoring
• Workflow: Evaluate
• Concept: Agentic evaluation