> For the complete documentation index, see [llms.txt](https://docs.layerlens.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.layerlens.ai/more-in-this-section-2/scenarios-6.md).
# Legal scenarios
Legal AI's defining failure mode is the **fabricated citation** — *Mata v. Avianca, Inc.* (S.D.N.Y. 2023) is the canonical reference for sanctioned attorney misconduct stemming from AI-generated hallucinations. Every scenario below treats verifiability as the first-order property.
## Contract review and analysis
**Setup:** AI reviews contracts to identify key clauses, flag non-standard terms, compare against a playbook, and suggest redlines.
**Why evaluation is hard:** material clauses can be buried in **cross-references** (e.g., Section 8.2(a) referencing Exhibit B, Paragraph 4) — the AI catches "unlimited liability" in plain text but misses the structural reference.
**Stakes:** missed unfavorable clauses (unlimited liability, auto-renewal, IP assignment); malpractice for law firms; financial exposure for in-house teams.
**Eval criteria:**
* Clause-recall scorer — every material clause from the attorney-annotated set must surface
* Cross-reference judge (GEPA-tuned ≥50 examples) — clauses defined via structural references are correctly resolved
* Risk-classification judge — risk-tier matches attorney verdict
* Hallucinated-clause-text rule — generated redline text must match clause-source text where quoted
## Legal research assistant
**Setup:** AI researches case law, statutes, and regulations to support brief drafting and answer legal questions.
**Why evaluation is hard:** *Mata v. Avianca* and successor cases mean **every fabricated citation in a filing is a sanctionable event** — regardless of whether anyone caught it before submission.
**Stakes:** court sanctions per *Mata*; bar discipline; malpractice exposure; reputational cycle (ABA Journal coverage of every public sanction).
**Eval criteria:**
* Citation-existence custom code grader — every cited case looks up against an authoritative legal database; any missing = fabricated. Hallucination rate must be 0% for the evaluation to pass.
* Holding-accuracy judge (GEPA-tuned ≥50 attorney-labeled examples — scored output) — does the cited case actually stand for the proposition the AI claims?
* Quote-accuracy scorer — quoted passages match the source verbatim
* Jurisdictional-relevance judge — citations are from the right court/jurisdiction for the brief
* Recency / overrule-detection scorer — never cites overruled cases (matched against a treatment database)
## Discovery / e-discovery review
**Setup:** AI reviews documents for relevance, privilege, and responsiveness in litigation discovery.
**Why evaluation is hard:** the AI may correctly process body text but miss content in **embedded tables and scanned attachments** — relevant material in a different modality slips through.
**Stakes:** spoliation sanctions ($10K-$1M+); privilege waiver (catastrophic disclosure); inability to defend production proportionality.
**Eval criteria:**
* Relevance recall scorer — must be >95% for defensibility
* Privilege-detection judge (GEPA-tuned ≥50 examples) — attorney-client and work-product detection accuracy
* Multimodal-coverage scorer — embedded tables, scanned attachments, image OCR captured (not skipped)
* PII / personal-data detection scorer — ≥99% for GDPR / CCPA contexts
* Inter-reviewer consistency scorer — Cohen's kappa or equivalent for reviewer agreement
## Contract / document automation
**Setup:** AI drafts contracts, NDAs, and legal documents from templates + case-specific inputs.
**Why evaluation is hard:** template substitution errors are easy; tone, voice, and jurisdictional appropriateness require subjective judgment.
**Stakes:** unenforceable contracts; embarrassing template artifacts ("\[CLIENT NAME]" in published doc); jurisdictional mismatches.
**Eval criteria:**
* Template-substitution scorer (custom code) — every placeholder filled; no leftover markup
* Jurisdictional-applicability judge — clauses match the governing-law jurisdiction
* Voice / tone judge — matches firm's house style
* Cross-reference integrity scorer — internal section references resolve correctly
## Compliance / regulatory analysis
**Setup:** AI analyzes regulatory texts, drafts compliance memos, and identifies obligations from new regulations.
**Why evaluation is hard:** regulatory text is dense, hierarchical, and context-dependent. Stale regulatory interpretation (citing superseded guidance) is a frequent failure.
**Eval criteria:**
* Regulation-currency rule — cited regulation version must match the active version (document-hash check)
* Obligation-extraction scorer — every actionable obligation identified, classified by deadline / who-must-act
* Faithfulness judge — every claim cites a specific regulation section
* Plain-language judge (GEPA-tuned) — output translates regulatory text appropriately for the audience
## See also
* [Industry pattern: AI research citation verification](/4.2-industry-use-cases/pattern-2.md) — these scenarios composed into one runnable evaluation
* [Eval patterns](/more-in-this-section-2/eval-patterns-6.md) | [Compliance](/more-in-this-section-2/compliance-6.md)
* [SDK sample: legal\_contracts.py](https://github.com/layerlens/stratix-python/blob/main/samples/industry/legal_contracts.py)
* [SDK sample: legal\_research.py](https://github.com/layerlens/stratix-python/blob/main/samples/industry/legal_research.py)
---
# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.
## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.
Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:
```
GET https://docs.layerlens.ai/more-in-this-section-2/scenarios-6.md?ask=&goal=
```
`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.
Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.