Build a trace set for agentic evals

Guide — build a trace set for agentic evaluation.

1. Identify representative inputs (happy paths, edge cases, known-bad).
2. Run your agent against each; capture as a trace.
3. Tag and group them into a dataset (a named, versioned trace set).
4. Use the dataset as input to agentic evaluation.

No traces yet? Generate them

You don't need production traffic to build a trace set. From Catalog → Datasets → New Dataset → Generate synthetic traces, produce a realistic multi-agent dataset from a built-in industry scenario (or by varying a few real traces) in minutes, then evaluate it like any other dataset. See Synthetic data.

See also

• Datasets
• Synthetic data
• Concept: Agentic evaluation
• Industry pattern: insurance missed escalation