Recipe: detect tool-call regressions in agents

Recipe — detect tool-call regressions. Pointer to the SDK code-review cowork sample.

Canonical samples:

• samples/cowork/code_review.py — Instrumentor-Reviewer pattern: one agent uploads code traces, another evaluates with code-execution, security, and metrics judges.
• samples/core/trace_investigation.py — surface errors, latency outliers, anomalies in trace data.

Pattern

Capture full agent traces including tool-call spans. Build a judge whose rubric forbids specific tool calls in specific contexts (e.g., destructive APIs only when input scope says allowed). Run via client.trace_evaluations.create(trace_id=, judge_id=). Compare to baseline via samples/core/compare_evaluations.py.

See also

• Industry pattern: insurance missed escalation
• Concept: Agentic evaluation