Government and public sector evaluation patterns

Government evaluation patterns — primitive ratio, span rules, GEPA labeling, cadence aligned to NIST AI RMF and OMB M-24-10.

Recommended primitive ratio for government AI: 60/25/15 — deterministic rules, code assertions, judges. Public-sector AI inherits the strictest documentation and reviewability burden of any vertical because every decision can be challenged in court or audit.

Primitive ratio

Primitive	Share	Why
Deterministic rules	60%	Statutory deadlines, exemption citations, eligibility rule references, FOIA exemptions
Code assertions	25%	Reading-level scorers, PII recall, routing-table validation, criterion coverage
LLM judges	15%	Plain-language quality, court-defensibility of exemption rationale, denial-reason readability

Pattern table

Scenario	Primary primitive	Specific check
Citizen chatbot — policy grounding	Judge	Faithfulness to current agency rule
Citizen chatbot — reading level	Code scorer	Flesch-Kincaid ≤ 8
Citizen chatbot — escalation	Rule	Eligibility/legal questions auto-escalate
Benefits — rule citation	Rule	Citation to CFR/state reg present
Benefits — recommendation gate	Rule	Output marked "recommendation" not "decision"
Benefits — fairness	Code scorer	Group-fairness ratio ≥ 0.8 across protected classes
Benefits — denial-reason quality	Judge	Plain-language readability for citizen
FOIA — exemption citation	Rule	Statutory subsection present per redaction
FOIA — deadline	Rule	20-day clock tracked per request
FOIA — PII recall	Code scorer	≥ 99% on labeled PII set
FOIA — over-redaction	Judge	Court-defensibility
Procurement — criterion coverage	Rule	All solicitation criteria scored
Procurement — fairness	Code scorer	Bias check across business categories
Public-safety — disparate impact	Code scorer	Required on any enforcement-influencing output
Public-safety — advisory gate	Rule	No automated enforcement actions

Span rules

• policy.lookup — version of the agency rule referenced
• eligibility.eval — which CFR/regulation sections were applied
• exemption.cite — FOIA exemption subsection per redaction
• routing.decide — destination office/form match
• bias.score — disparate impact tracked per output

NIST AI RMF mapping

The four NIST AI Risk Management Framework functions map to evaluation evidence:

NIST function	Stratix evidence
GOVERN	Tier policy + RBAC + audit trail per evaluation
MAP	AI inventory tied to scenario suite per use case
MEASURE	Evaluation reports + drift dashboards
MANAGE	Alert engine + rule pass-rate threshold gates

OMB M-24-10 alignment

For federal agencies covered by OMB M-24-10 (Advancing Governance, Innovation, and Risk Management for Agency Use of AI):

• Rights-impacting and safety-impacting AI must have pre- and post-deployment testing
• Continuous monitoring with documented thresholds
• Public AI use case inventories
• Generative-AI-specific guardrails

Stratix evaluation evidence shape:

• Pre- and post-deployment scenario suite frozen at deployment time
• Production traces sampled and scored against same suite
• Quarterly reports on drift, fairness, accuracy

GEPA labeling

For each government judge, label ≥ 50 paired examples:

• Citizen-services chatbot: Stratified across program types and languages
• Benefits eligibility: Mix of approval/denial/needs-more-info, labeled by senior caseworkers
• FOIA exemption: Labeled by FOIA officers; exemption rulings from FOIAonline / state appeals
• Procurement: Labeled by contracting officers
• Public-safety: Labeled by trained civil-rights reviewers, not just enforcement personnel

Run judge optimization with budget="medium". Hold out 20% for non-overlapping validation.

Cadence

Cadence	Activity
Per-PR	CI gate runs scenario suite (rules + assertions)
Daily	Sample production traces; full evaluation including judges
Weekly	Disparate-impact audit; FOIA deadline-compliance report
Monthly	GEPA re-optimization on judge drift
Quarterly	NIST AI RMF MEASURE function report; OMB M-24-10 reporting prep
Annually	Public AI use-case inventory update; full bias re-audit

See also

• Government scenarios
• Government compliance
• Recipe: GEPA basics