> For the complete documentation index, see [llms.txt](https://docs.layerlens.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.layerlens.ai/more-in-this-section-2/eval-patterns-4.md).

# Healthcare evaluation patterns

Healthcare evaluation patterns — common scorer, rule, and judge combinations.

A field guide to which Stratix primitive to reach for in clinical-AI scenarios. The pattern for healthcare differs from other industries because of the asymmetric cost of errors: **a deterministic rule that catches one bad output is worth more than a judge that grades fifty good ones.**

## Rule of thumb: 70 / 20 / 10

For a typical healthcare evaluation:

* **\~70% deterministic rules** — the bulk of safety enforcement
* **\~20% natural-language assertions** — fuzzy correctness (cited correctly? appropriate care level?)
* **\~10% LLM judges** — residual subjective dimensions (tone, plain-language quality)

Inverting this ratio (judge-heavy) is the most common mistake.

## Pattern table

| Dimension                     | Approach                                         | Notes                                                                    |
| ----------------------------- | ------------------------------------------------ | ------------------------------------------------------------------------ |
| Clinical-claim faithfulness   | LLM judge, GEPA-optimized                        | Tune against ≥50 clinician-labeled examples; hold out ≥20 for validation |
| Citation existence            | Deterministic rule                               | Cited section IDs must resolve to real chunks                            |
| Dose-recommendation surfacing | Deterministic rule + numeric scorer              | Extract every dose mention; require exact match to source                |
| Drug-interaction coverage     | Deterministic rule                               | Every drug pair in input must trigger a check                            |
| Emergency escalation          | Deterministic rule (regex / classifier)          | Hard floor below LLM judgment                                            |
| Reasoning soundness           | LLM judge, GEPA-optimized                        | Penalize correct-answer-by-flawed-reasoning                              |
| PHI redaction                 | Deterministic rule + regex                       | Block unredacted PHI in any logged output                                |
| Care-level appropriateness    | LLM judge                                        | Triage scenarios                                                         |
| Empathy / patient-facing tone | LLM judge                                        | Use sparingly                                                            |
| Reading level                 | Deterministic scorer (Flesch-Kincaid or similar) | Required for patient-facing                                              |
| Required-field presence       | Custom code grader                               | Diagnoses, medications, allergies, vitals                                |

## Span-level rules

Healthcare agents often need rules at the span level — examples:

* **Retrieval span:** "the retrieval span must not return content from outside the approved guideline corpus."
* **LLM span:** "the prescribing span must include a citation to a prescribing reference for any dose mentioned."
* **Tool span:** "the EHR write-back span may only fire for the patient ID in `inputs`."

Stratix's span-level scoring makes these expressible. See [span-level scoring recipe](/more-in-this-section-6/span-level-scoring.md).

## Trace-set curation

A healthcare trace set should over-sample edge cases:

* Allergy combinations
* Pediatric / geriatric weight-based dosing
* Pregnancy / lactation contraindications
* Renal / hepatic impairment dosing
* Crisis scenarios (suicidal ideation, anaphylaxis, MI symptoms)
* Multilingual presentations
* Low-health-literacy phrasings

A 50-trace curated set tilted toward edge cases catches more failures than a 500-trace random sample.

## GEPA labeling for clinical judges

Clinicians label inconsistently across specialties; calibrate before optimizing.

1. Have ≥3 clinicians label the same 30 examples
2. Compute inter-rater agreement (Cohen's kappa or similar)
3. Discuss disagreements; converge on a rubric
4. Have those clinicians label 100+ examples with the converged rubric
5. GEPA-optimize the judge against the consensus labels

Skipping the calibration step leaves you tuning a judge to noise.

## Continuous evaluation cadence

Production healthcare AI should run trace evaluations:

* **Per-PR** on a curated 50-trace gating set
* **Daily** on a sampled production slice
* **Weekly** full review with clinical leadership of any drift

## See also

* [Scenarios](/more-in-this-section-2/scenarios-4.md)
* [Compliance](/more-in-this-section-2/compliance-4.md)
* [Cookbook: clinical Q\&A](/more-in-this-section-2/industry-healthcare-clinical-qa.md)
* [Tutorial 5: GEPA optimize](/9.-improve-tune-the-system/05-gepa-optimize.md)