Shareable benchmark table
Controlled campaign design
Frozen dimensions of the paired 12-audit, three-replicate comparison.
Table 05 · Frozen protocol
Controlled campaign design
| Dimension | Frozen design |
|---|---|
| Systems | GPT-5.6 Sol · High via Codex CLI; Grok 4.6 Build · High via Grok CLI |
| Sample | 12 audits × 3 replicates × 2 systems = 72 assignments |
| User input | Byte-identical prompt and deterministic source bundle within each paired block |
| Access | No task tools, shell, web, browser, plugins, skills, memory, workflows, or subagents |
| Runtime | Fresh workspace and session; sequential runs on the same Apple Silicon host |
| Timeout | 1,800 seconds from launch to final response |
| Output | One final Markdown audit report captured by the harness |
Provider system prompts, tokenizers, CLIs, compaction, safety routing, and cloud backends remain system-level differences.
