> For the complete documentation index, see [llms.txt](https://docs.layerlens.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.layerlens.ai/more-in-this-section-9/commands.md).

# CLI — Command Reference

Complete reference for every stratix CLI command — trace, judge, evaluate, scorer, space, bulk, ci.

Complete reference for all `stratix` CLI commands.

## Command tree

```
stratix [global-options]
├── trace
│ ├── list List traces
│ ├── get Get a trace by ID
│ ├── search Search traces
│ ├── export Export a trace as JSON
│ └── delete Delete a trace
├── judge
│ ├── list List judges
│ ├── get Get a judge by ID
│ ├── create Create a new judge
│ └── test Test a judge against a trace
├── evaluate
│ ├── list List evaluations
│ ├── get Get an evaluation by ID
│ └── run Run a new evaluation
├── integration
│ ├── list List integrations
│ └── test Test an integration
├── scorer
│ ├── list List scorers
│ ├── get Get a scorer by ID
│ ├── create Create a new scorer
│ └── delete Delete a scorer
├── space
│ ├── list List evaluation spaces
│ ├── get Get a space by ID or slug
│ ├── create Create a new space
│ └── delete Delete a space
├── bulk
│ └── eval Run evaluations in bulk
├── ci
│ └── report Generate a CI summary report
└── completion Print shell completion setup
```

***

## Global options

These options are available on every command:

```
--api-key TEXT API key (env: LAYERLENS_STRATIX_API_KEY)
--host TEXT API host
--port INTEGER API port
--format [table|json] Output format (default: table)
--verbose, -v Enable debug output
--version Show version and exit
--help Show help and exit
```

***

## trace

Manage traces.

### `trace list`

List traces with optional filtering and pagination.

```bash
stratix trace list [OPTIONS]
```

| Option         | Type     | Description      |
| -------------- | -------- | ---------------- |
| `--page`       | int      | Page number      |
| `--page-size`  | int      | Results per page |
| `--source`     | text     | Filter by source |
| `--status`     | text     | Filter by status |
| `--sort-by`    | text     | Sort field       |
| `--sort-order` | asc/desc | Sort order       |

### `trace get`

Get a single trace by ID.

```bash
stratix trace get <ID>
```

### `trace search`

Search traces by query string.

```bash
stratix trace search <QUERY> [OPTIONS]
```

| Option         | Type     | Description      |
| -------------- | -------- | ---------------- |
| `--page`       | int      | Page number      |
| `--page-size`  | int      | Results per page |
| `--source`     | text     | Filter by source |
| `--status`     | text     | Filter by status |
| `--sort-by`    | text     | Sort field       |
| `--sort-order` | asc/desc | Sort order       |

### `trace export`

Export a trace as JSON.

```bash
stratix trace export <ID> [OPTIONS]
```

| Option           | Type | Description                   |
| ---------------- | ---- | ----------------------------- |
| `--output`, `-o` | path | Output file (default: stdout) |

### `trace delete`

Delete a trace by ID.

```bash
stratix trace delete <ID> [OPTIONS]
```

| Option        | Type | Description              |
| ------------- | ---- | ------------------------ |
| `--yes`, `-y` | flag | Skip confirmation prompt |

***

## judge

Manage judges.

### `judge list`

```bash
stratix judge list [OPTIONS]
```

| Option        | Type | Description      |
| ------------- | ---- | ---------------- |
| `--page`      | int  | Page number      |
| `--page-size` | int  | Results per page |

### `judge get`

```bash
stratix judge get <ID>
```

### `judge create`

Create a new judge.

```bash
stratix judge create [OPTIONS]
```

| Option       | Type | Required | Description                 |
| ------------ | ---- | -------- | --------------------------- |
| `--name`     | text | yes      | Judge name                  |
| `--goal`     | text | yes      | Evaluation goal description |
| `--model-id` | text | no       | Model ID for the judge      |

### `judge test`

Test a judge by running it against a trace. Creates a trace evaluation.

```bash
stratix judge test [OPTIONS]
```

| Option       | Type | Required | Description          |
| ------------ | ---- | -------- | -------------------- |
| `--judge-id` | text | yes      | Judge ID to test     |
| `--trace-id` | text | yes      | Trace ID to evaluate |

***

## evaluate

Manage evaluations.

### `evaluate list`

```bash
stratix evaluate list [OPTIONS]
```

| Option        | Type                                     | Description                                    |
| ------------- | ---------------------------------------- | ---------------------------------------------- |
| `--page`      | int                                      | Page number                                    |
| `--page-size` | int                                      | Results per page                               |
| `--status`    | text                                     | Filter: pending, in-progress, success, failure |
| `--sort-by`   | submitted\_at/accuracy/average\_duration | Sort field                                     |
| `--order`     | asc/desc                                 | Sort order                                     |

### `evaluate get`

```bash
stratix evaluate get <ID>
```

### `evaluate run`

Run a new evaluation. Accepts model/benchmark by ID, key, or name.

```bash
stratix evaluate run [OPTIONS]
```

| Option        | Type | Required | Description                     |
| ------------- | ---- | -------- | ------------------------------- |
| `--model`     | text | yes      | Model ID, key, or name          |
| `--benchmark` | text | yes      | Benchmark ID, key, or name      |
| `--wait`      | flag | no       | Wait for evaluation to complete |

***

## integration

Manage integrations.

### `integration list`

```bash
stratix integration list [OPTIONS]
```

| Option        | Type | Description      |
| ------------- | ---- | ---------------- |
| `--page`      | int  | Page number      |
| `--page-size` | int  | Results per page |

### `integration test`

Test an integration by ID.

```bash
stratix integration test <ID>
```

***

## scorer

Manage scorers.

### `scorer list`

```bash
stratix scorer list [OPTIONS]
```

| Option        | Type | Description      |
| ------------- | ---- | ---------------- |
| `--page`      | int  | Page number      |
| `--page-size` | int  | Results per page |

### `scorer get`

```bash
stratix scorer get <ID>
```

### `scorer create`

```bash
stratix scorer create [OPTIONS]
```

| Option          | Type | Required | Description                     |
| --------------- | ---- | -------- | ------------------------------- |
| `--name`        | text | yes      | Name (3–64 characters)          |
| `--description` | text | yes      | Description (10–500 characters) |
| `--model-id`    | text | yes      | Model ID for scoring            |
| `--prompt`      | text | yes      | Scoring prompt                  |
| `--dry-run`     | flag | no       | Preview without executing       |

### `scorer delete`

```bash
stratix scorer delete <ID> [OPTIONS]
```

| Option        | Type | Description               |
| ------------- | ---- | ------------------------- |
| `--yes`, `-y` | flag | Skip confirmation prompt  |
| `--dry-run`   | flag | Preview without executing |

***

## space

Manage evaluation spaces.

### `space list`

```bash
stratix space list [OPTIONS]
```

| Option        | Type     | Description                           |
| ------------- | -------- | ------------------------------------- |
| `--page`      | int      | Page number                           |
| `--page-size` | int      | Results per page                      |
| `--sort-by`   | text     | Sort field (e.g. weight, created\_at) |
| `--order`     | asc/desc | Sort order                            |

### `space get`

```bash
stratix space get <ID>
```

Accepts an ID or slug.

### `space create`

```bash
stratix space create [OPTIONS]
```

| Option          | Type                  | Required | Description                      |
| --------------- | --------------------- | -------- | -------------------------------- |
| `--name`        | text                  | yes      | Space name                       |
| `--description` | text                  | no       | Description (max 500 characters) |
| `--visibility`  | private/public/tenant | no       | Visibility level                 |
| `--dry-run`     | flag                  | no       | Preview without executing        |

### `space delete`

```bash
stratix space delete <ID> [OPTIONS]
```

| Option        | Type | Description               |
| ------------- | ---- | ------------------------- |
| `--yes`, `-y` | flag | Skip confirmation prompt  |
| `--dry-run`   | flag | Preview without executing |

***

## bulk

Bulk operations.

### `bulk eval`

Run evaluations in bulk. Supports two modes:

**Mode 1: JSONL file**

```bash
stratix bulk eval --file jobs.jsonl [OPTIONS]
```

Each line in the JSONL file is a JSON object with `model` and `benchmark` fields:

```json
{"model": "openai/gpt-4o", "benchmark": "arc-agi-2"}
{"model": "anthropic/claude-3-opus", "benchmark": "arc-agi-2"}
```

**Mode 2: Single model + benchmark**

```bash
stratix bulk eval --model openai/gpt-4o --benchmark arc-agi-2 --wait [OPTIONS]
```

**Mode 3: Judge + trace IDs**

```bash
stratix bulk eval --judge-id <JUDGE_ID> --traces trace_ids.txt [OPTIONS]
```

The traces file contains one trace ID per line.

| Option        | Type | Description                          |
| ------------- | ---- | ------------------------------------ |
| `--file`      | path | JSONL file with evaluation jobs      |
| `--model`     | text | Model ID/name (use with --benchmark) |
| `--benchmark` | text | Benchmark ID/name (use with --model) |
| `--judge-id`  | text | Judge ID (use with --traces)         |
| `--traces`    | path | File with trace IDs (one per line)   |
| `--dry-run`   | flag | Preview without executing            |
| `--wait`      | flag | Wait for all evaluations to complete |

***

## ci

CI/CD pipeline helpers.

### `ci report`

Generate a markdown summary of recent evaluations, suitable for GitHub Actions job summaries.

```bash
stratix ci report [OPTIONS]
```

| Option           | Type | Description                                    |
| ---------------- | ---- | ---------------------------------------------- |
| `--output`, `-o` | path | Output file (default: stdout)                  |
| `--limit`        | int  | Number of evaluations to include (default: 10) |
| `--dry-run`      | flag | Preview without fetching data                  |

***

## completion

Print shell completion setup instructions.

```bash
stratix completion <SHELL>
```

Where `SHELL` is one of: `bash`, `zsh`, `fish`, `powershell`.