Post-release continuous evaluation

Recipe — post-release continuous evaluation on production traces.

After release, score recent production traces on a recurring cadence (hourly / daily). Stratix's SDK doesn't ship a scheduler — wire the per-trace client.trace_evaluations.create(...) call into your own scheduled job.

Per-run pattern

from layerlens import Stratix
client = Stratix()

# Pull a sample of recent production traces
recent = client.traces.get_many(
 page=1,
 page_size=50,
 sort_by="created_at",
 sort_order="desc",
 # plus any source / judge_id / search filters relevant to your feature
)

# Score each with your continuous-eval judge
import asyncio
from layerlens import AsyncStratix

async def score_batch():
 aclient = AsyncStratix()
 creates = [aclient.trace_evaluations.create(trace_id=t.id, judge_id=CONTINUOUS_JUDGE_ID)
 for t in recent.traces]
 evals = await asyncio.gather(*creates)
 return await asyncio.gather(*[aclient.trace_evaluations.wait_for_completion(e.id) for e in evals])

results = asyncio.run(score_batch())
fails = [r for r in results if not r.passed]
if len(fails) / max(len(results), 1) > 0.05:
 alert_oncall(fails)

Wire to a scheduler

Run this script from a GitHub Actions schedule, AWS EventBridge → Lambda, k8s CronJob, or any scheduler — same pattern as daily smoke eval.

See also

• Concept: Continuous evaluation
• Tutorial 4: Score live traces