First evaluation via SDK

Run your first benchmark evaluation programmatically via the Stratix Python SDK.

Twenty lines of Python that run a real evaluation.

Prerequisites

• SDK installed
• LAYERLENS_STRATIX_API_KEY exported
• A Stratix Premium account with starter ECU

Sync version

from layerlens import Stratix

client = Stratix()

# Pick a model and a benchmark from the public catalog
model = client.models.get_by_key("openai/gpt-4o")
benchmark = client.benchmarks.get_by_key("arc-agi-2")

# Create the evaluation
evaluation = client.evaluations.create(
 model=model,
 benchmark=benchmark,
)

# Wait for completion (polling)
result = client.evaluations.wait_for_completion(evaluation)
print(f"Accuracy: {result.accuracy}")

Async version

Every method has an async counterpart on AsyncStratix:

import asyncio
from layerlens import AsyncStratix

async def main():
 client = AsyncStratix()
 model = await client.models.get_by_key("openai/gpt-4o")
 benchmark = await client.benchmarks.get_by_key("arc-agi-2")
 evaluation = await client.evaluations.create(model=model, benchmark=benchmark)
 result = await client.evaluations.wait_for_completion(evaluation)
 print(f"Accuracy: {result.accuracy}")

asyncio.run(main())

What just happened

1. The SDK authenticated with your API key.
2. It fetched the model and benchmark records from the public catalog.
3. It created an evaluation in your org.
4. The platform queued the work; the SDK polled for completion.
5. The result was returned; you got an accuracy score.

Verify

The same evaluation should appear in Premium → Evaluations. Same record, same ID, same scores.

Try variations

Different model

model = client.models.get_by_key("anthropic/claude-opus-4-7")

Different benchmark

benchmark = client.benchmarks.get_by_key("mmlu")

Browse public results before running

public_models = client.public.models.get()
for m in public_models.models[:5]:
 print(m.key, m.name)

Error handling

from layerlens import Stratix, NotFoundError, AuthenticationError, APIError

try:
 model = client.models.get_by_id("nonexistent-id")
except NotFoundError as e:
 print(f"Not found: {e.message}")
except AuthenticationError as e:
 print(f"Auth failed: {e.message}")
except APIError as e:
 print(f"API error ({e.status_code}): {e.message}")

Where to next

• Python SDK reference
• Tutorial: First evaluation in 10 minutes
• Cookbook recipes