> For the complete documentation index, see [llms.txt](https://docs.layerlens.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.layerlens.ai/4.1-general-use-cases/rag-evaluation.md).

# RAG evaluation

Evaluate RAG pipelines end-to-end — retrieval quality, faithfulness, and answer quality.

A RAG (retrieval-augmented generation) pipeline has more failure modes than a plain LLM call. Bad retrieval kills good models; great retrieval can't save a hallucinating one. RAG evaluation scores each stage so you know where the failure actually is.

## The shape of the work

1. **Define the three dimensions.**

* **Retrieval quality** — did we get back relevant chunks?
* **Faithfulness** — did the answer ground itself in the retrieved chunks (not invent)?
* **Answer quality** — did the final answer actually help the user?

2. **Pick scorers and judges.**

* Retrieval: classical IR metrics (precision\@k, recall\@k) as scorers; or an LLM judge that grades chunk relevance.
* Faithfulness: an LLM judge that compares answer claims against the retrieved chunks. GEPA-optimize against a labeled set.
* Answer quality: an LLM judge for end-to-end helpfulness, again GEPA-optimized.

3. **Run as a trace evaluation.** Your pipeline emits a trace with retrieval span + generation span. Stratix grades each.
4. **Compare configurations.** Try a different retriever, a different chunk size, a different model — re-run, see the score deltas per dimension.
5. **Lock in CI gates.** Faithfulness regressions are particularly costly; gate them.

## Why it works on Stratix

* **Trace-first.** Multi-span traces are first-class; you can score retrieval and generation separately.
* **Judge optimization.** GEPA tunes judges to match human labels for faithfulness — usually the trickiest dimension.
* **Compare-models.** Try the same retrieval feeding three different generators; see which generator is most faithful to the same chunks.

## Tools you'll use

* [Stratix Premium — Agent Evaluation](/7.-observe-see-whats-happening/agent-evaluation.md) (RAG is a small agent)
* [Stratix Premium — Traces](/7.-observe-see-whats-happening/traces.md)
* [Stratix Premium — Judges](/8.-evaluate-score-the-outputs/judges.md)
* [SDK trace ingestion](/4.1-general-use-cases/general.md)

## Outcomes you should see

You'll know this is working when:

* **Faithfulness judge agreement with humans crosses 90%** after GEPA optimization.
* **Retrieval and answer scores diverge meaningfully** — when answer quality drops, you can tell whether retrieval or generation is at fault.
* **Hallucination rate trends downward** over consecutive releases, not just bounces.
* **Per-chunk-strategy comparisons take <1 day** instead of multi-week experiments.

## Anti-patterns

* **Scoring only the final answer.** You'll know quality dropped; you won't know which stage caused it.
* **Faithfulness judge without optimization.** Out-of-the-box LLM judges over-credit confident-sounding hallucinations. GEPA-optimize against a labeled set.
* **No retrieval ground truth.** If you can't say "these are the right chunks," your retrieval scores are just plausibility scores.

## Where to next

* [Concept: Judges (faithfulness)](/8.-evaluate-score-the-outputs/judges-1.md)
* [Concept: Traces and spans](/6.-build-wire-your-code/traces-and-spans.md)
* [Cookbook: RAG recipes](/2.-get-started/all-cookbook-recipes.md)