> For the complete documentation index, see [llms.txt](https://docs.layerlens.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.layerlens.ai/more-in-this-section-2/eval-patterns-2.md).

# Financial-services evaluation patterns

Financial-services evaluation patterns — primitive ratios, dimensions, fair-lending considerations, GEPA labeling, continuous cadence.

The asymmetric-cost shape of financial-services AI drives the primitive ratio. Detection accuracy is necessary but not sufficient — fair-treatment and disclosure-presence dimensions are where regulatory exposure lives.

## Rule of thumb: 60 / 25 / 15

For a typical financial-services evaluation:

* **\~60% deterministic rules / scorers** — sanctions coverage, disclaimer presence, position limits, PII handling, no-specific-recommendation patterns
* **\~25% natural-language assertions and code-based fairness scorers** — disparity computation, suitability matching, document-extraction accuracy
* **\~15% LLM judges** — explanation quality (SAR narration, customer-facing rationale), suitability judgment in ambiguous cases

Inverting this ratio (judge-heavy) is the most common mistake.

## Pattern table

| Dimension                        | Approach                                    | Notes                                                                      |
| -------------------------------- | ------------------------------------------- | -------------------------------------------------------------------------- |
| Detection precision/recall       | Custom code grader                          | Against labeled fraud / sanctions / SAR datasets                           |
| False-positive disparity         | Custom code grader                          | Per-segment (ZIP, surname-derived demographic, age band) — 1.25× threshold |
| Sanctions coverage               | Hard rule                                   | Recall must be 100%; missing checks = CRITICAL                             |
| Suitability matching             | LLM judge, GEPA-tuned per risk-profile band | ≥50 labels per band                                                        |
| Required-disclaimer presence     | Substring scorer                            | FINRA / SOX / TILA disclosures                                             |
| No-specific-securities rule      | Regex / classifier                          | Customer-facing AI never recommends individual securities                  |
| Document-extraction accuracy     | Custom code grader                          | Field-by-field exact match                                                 |
| Cross-document consistency       | LLM judge                                   | Values match across application + supporting docs                          |
| Numeric / dollar-amount accuracy | Custom code grader                          | Critical for income, balances, transaction amounts                         |
| PII redaction                    | Regex rule                                  | No SSN / account number echo in any logged span                            |
| Adverse-media relevance          | LLM judge                                   | Per match — keep precision high to limit reviewer overload                 |
| Latency under SLA                | Latency scorer                              | Real-time fraud decisions need <300ms p95                                  |

## Span-level rules

Span-level rules matter for compound workflows:

* **Feature-extraction span:** sensitive fields (SSN, full account number) must be hashed before scoring.
* **Model-scoring span:** every score above the SAR threshold must include the contributing features.
* **Explanation span:** every explanation must cite specific transaction or customer features (no opaque rationale).

## GEPA labeling for financial-services judges

Suitability and SAR-narration judges benefit substantially from GEPA — but the labeling needs careful calibration.

1. Label by **multi-reviewer agreement** — at least one compliance officer + one product reviewer per example
2. Stratify labeling by risk-profile band (suitability) or by transaction shape (SAR)
3. Hold out 20% as validation
4. Re-optimize quarterly as regulator interpretive guidance shifts

## Trace-set curation

Over-sample edge cases:

* High-net-worth conservative profiles paired with growth-product recommendations
* Sanctions-adjacent names (close to OFAC list but not on it)
* Self-employment income with depreciation add-backs
* Identical-amount transactions across multiple devices (account takeover signal)
* Cross-border transactions where both endpoints might be sanctioned
* Names that are PEP-adjacent in non-Latin scripts

## Continuous evaluation cadence

* **Per-PR** on the regression suite (small, fast)
* **Hourly** on a 1% sample of production fraud decisions during business hours
* **Daily** on the full SAR-eligible decision set
* **Quarterly** disparity audit with compliance signoff

## See also

* [Scenarios](/more-in-this-section-2/scenarios-2.md) | [Compliance](/more-in-this-section-2/compliance-2.md)
* [Industry pattern: fair-lending platform pattern](/4.2-industry-use-cases/pattern-1.md)
* [SDK sample: financial\_fraud.py](https://github.com/layerlens/stratix-python/blob/main/samples/industry/financial_fraud.py)