Education and EdTech evaluation patterns

Education evaluation patterns — primitive ratio, span rules, GEPA labeling, cadence.

Recommended primitive ratio for education AI: 50/30/20 — deterministic rules, code assertions, judges. Subject-matter accuracy and pedagogical-quality assessment lift the judge share above what other regulated industries use.

Primitive ratio

Primitive	Share	Why
Deterministic rules	50%	Direct-answer guards, instructor-final-decision, confidence thresholds, accommodation accuracy
Code assertions	30%	Inter-rater agreement, fairness, reading-level, captioning WER
LLM judges	20%	Pedagogical scaffolding, alternate-format faithfulness, representation diversity

Pattern table

Scenario	Primary primitive	Specific check
Tutoring — accuracy	Judge	Subject-matter faithfulness
Tutoring — scaffolding	Judge	Asks-before-tells rubric
Tutoring — direct-answer guard	Rule	No outright answers on protected assignments
Tutoring — age appropriate	Code scorer	Reading level matches grade
Grading — inter-rater agreement	Code assertion	Cohen's kappa vs. human reference
Grading — demographic fairness	Code scorer	Score parity across proxies
Grading — rubric coverage	Rule	Every criterion scored
Grading — instructor approval	Rule	Final grades require instructor sign-off
Curriculum — standards citation	Rule	Items cite alignment standard
Curriculum — accommodation accuracy	Code assertion	Match IEP/504 database
Curriculum — representation	Code scorer	Diversity of examples
Integrity — confidence threshold	Rule	AI never unilateral
Integrity — disparate impact	Code scorer	Group fairness in flagging
Integrity — evidence citation	Rule	Specific evidence required
Accessibility — faithfulness	Judge	Alternate format preserves content
Accessibility — captioning WER	Code assertion	Below regulatory threshold

Span rules

• student.context.lookup — grade, IEP/504, language preference
• standards.cite — Common Core / NGSS / state / AP alignment
• rubric.coverage — every criterion scored
• accommodation.match — IEP/504 plan applicability
• integrity.flag — confidence and evidence chain
• instructor.review — approval state

GEPA labeling

For each education judge, label ≥ 50 paired examples (more for high-stakes):

• Pedagogical scaffolding: Curriculum specialists + classroom teachers; cover Socratic vs. directive responses
• Subject-matter faithfulness: Domain experts per subject (math/science/humanities/CS); ≥ 100 for STEM grading
• Accessibility faithfulness: Disability-services specialists label
• Representation diversity: Equity specialists label

Run judge optimization with budget="medium". For grading judges that affect student records, run with budget="heavy" and hold out 30%.

Cadence

Cadence	Activity
Per-deploy	CI gate runs scenario suite (rules + assertions)
Per-class-session	Sample tutoring traces; daily summary to teacher
Daily	Sample production traces across all scenarios
Weekly	Demographic-fairness audit on grading; integrity-flag overturn rate
Monthly	GEPA re-optimization on judges that have drifted
Quarterly	Standards-alignment refresh; accommodation database refresh
Annually	OCR / state-audit prep; full bias audit

See also

• Education scenarios
• Education compliance
• Recipe: GEPA basics