> For the complete documentation index, see [llms.txt](https://docs.layerlens.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.layerlens.ai/more-in-this-section-9/evaluations-1.md).

# Evaluations

client.evaluations — create benchmark and dataset evaluations, wait for completion, list, retrieve results.

## Create

The canonical pattern is to **fetch model and benchmark objects first, then pass them to `create`** (not just IDs):

```python
from layerlens import Stratix

client = Stratix()

# Fetch the objects you want to evaluate
model = client.models.get_by_key("openai/gpt-4o")
benchmark = client.benchmarks.get_by_key("arc-agi-2")

# Create the evaluation
evaluation = client.evaluations.create(
 model=model,
 benchmark=benchmark,
)
print(f"Created evaluation {evaluation.id}, status={evaluation.status}")
```

`get_by_key` is case-sensitive. Alternatively, fetch a list and pick:

```python
models = client.models.get(type="public", name="gpt-4o")
model = models[0]
```

## Wait for completion

Pass the **evaluation object** (not just the ID) to `wait_for_completion`:

```python
evaluation = client.evaluations.wait_for_completion(
 evaluation,
 interval_seconds=10,
 timeout_seconds=600, # 10 minutes
)
print(f"Evaluation {evaluation.id} finished with status={evaluation.status}")
```

Once the evaluation completes, retrieve results:

```python
if evaluation.is_success:
 results = client.results.get(evaluation=evaluation)
 print("Results:", results)
else:
 print(f"Evaluation did not succeed: {evaluation.status}")
```

The `Evaluation` object exposes `.is_success`, `.status`, `.id`, and other fields directly.

## List and filter

```python
# Get many evaluations
response = client.evaluations.get_many()

# Filter by status, accuracy, date — see samples/core/evaluation_filtering.py
filtered = client.evaluations.get_many(
 status="completed",
 sort_by="created_at",
 sort_order="desc",
 page_size=20,
)
```

## Async

Every method has an awaitable counterpart on `AsyncStratix`:

```python
import asyncio
from layerlens import AsyncStratix

async def main():
 client = AsyncStratix()
 model = await client.models.get_by_key("openai/gpt-4o")
 benchmark = await client.benchmarks.get_by_key("arc-agi-2")
 evaluation = await client.evaluations.create(model=model, benchmark=benchmark)
 result = await client.evaluations.wait_for_completion(evaluation)
 print(f"Accuracy: {result.accuracy}")

asyncio.run(main())
```

For concurrent runs, see `samples/core/async_workflow.py`.

## Result fields

The result object exposes:

* `accuracy` — overall accuracy for benchmark-style evaluations
* `status` — `"queued"`, `"running"`, `"completed"`, `"failed"`, `"cancelled"`
* `is_success` — boolean shortcut for `status == "completed"`
* `id` — stable evaluation ID

For per-row data and detailed scores, retrieve via `client.results.get(evaluation=evaluation)`.

## Compare

```python
# Compare results across multiple evaluation runs — see samples/core/compare_evaluations.py
```

## Source samples

| Sample                                 | What it shows                   |
| -------------------------------------- | ------------------------------- |
| `samples/core/quickstart.py`           | Minimal end-to-end              |
| `samples/core/run_evaluation.py`       | Full evaluation lifecycle       |
| `samples/core/benchmark_evaluation.py` | Model vs. benchmark             |
| `samples/core/async_workflow.py`       | Concurrent evaluations          |
| `samples/core/evaluation_filtering.py` | Filter and paginate             |
| `samples/core/compare_evaluations.py`  | Compare runs                    |
| `samples/core/evaluation_pipeline.py`  | Chain judges + traces + results |

## See also

* [Concept: Evaluations](/more-in-this-section-9/evaluations-1.md)
* [Tutorial 1: First evaluation](/8.-evaluate-score-the-outputs/01-first-evaluation.md)
* [Trace evaluations](/more-in-this-section-9/trace-evaluations-1.md) — separate path for evaluating traces with judges
* [Results](/more-in-this-section-9/results.md) — retrieve scored data after `wait_for_completion`