Media and entertainment evaluation patterns

Media and entertainment evaluation patterns — primitive ratio, span rules, GEPA labeling, cadence.

Recommended primitive ratio for media AI: 45/30/25 — deterministic rules, code assertions, judges. Media has more open-text creative output than other regulated industries, so judges weigh more.

Primitive ratio

Primitive	Share	Why
Deterministic rules	45%	Catalog freshness, rating/age gates, CSAM routing, quote fidelity, editor-review gates
Code assertions	30%	Per-category precision/recall, fairness, originality, GARM categories
LLM judges	25%	Faithfulness, tone/framing, row-title quality, summarization fidelity

Pattern table

Scenario	Primary primitive	Specific check
Recommendation — catalog freshness	Rule	Item live in region
Recommendation — rating gate	Rule	Item rating ≤ profile permission
Recommendation — diversity	Code scorer	Set diversity threshold
Recommendation — row-title quality	Judge	Title describes the row
Moderation — CSAM	Rule	100% NCMEC routing
Moderation — category accuracy	Code scorer	Per-policy P/R
Moderation — fairness	Code scorer	Action-rate parity
Moderation — coordination detection	Code scorer	Cluster signals
Ad — GARM accuracy	Code scorer	Per-category precision
Ad — exclusion adjacency	Rule	Advertiser exclusions enforced
News — faithfulness	Judge	Source-grounded summary
News — quote fidelity	Rule	Verbatim quote match
News — editor gate	Rule	Editor sign-off before publish
IP — rights citation	Rule	Above-threshold matches cite holder
IP — counter-notice routing	Rule	Human-review path

Span rules

Media traces should expose:

• catalog.lookup — region-availability and rating
• recommend.score — feature inputs and ranker version
• moderation.classify — per-category scores; routing decision
• ad.match — GARM categories and brand-safety bits
• editorial.review — editor identity and approval state
• rights.match — rights-holder match and confidence

GEPA labeling

For each media judge, label ≥ 50 paired examples:

• Recommendation row title: Trust-and-safety + product-content team labels good vs. clickbait vs. wrong
• Moderation per-category: Policy team labels; multiple reviewers with reconciliation; ≥ 200 examples for sensitive categories
• News summarization: Newsroom labels; cover paraphrase, ambiguous source, multi-source synthesis
• Brand safety: Sales + ops team labels per GARM; refresh quarterly

Run judge optimization with budget="medium" for routine judges, budget="heavy" for safety-critical (moderation, brand safety). Hold out 20%.

Cadence

Cadence	Activity
Per-deploy	CI gate runs scenario suite (rules + assertions)
Continuous	Catalog-freshness checks against live SKU/title feed
Hourly	Moderation drift on a sliding sample
Daily	Sample production traces; full scoring including judges
Weekly	Fairness audit; appeal-overturn rate; advertiser brand-safety report
Monthly	GEPA re-optimization on judge drift
Quarterly	DSA / OSA reporting prep; rights-detection false-positive audit

See also

• Media scenarios
• Media compliance
• Recipe: GEPA basics