Table library

The numbers, ready to reuse.

Every table has a stable share URL, responsive embed, CSV export, and pixel-faithful PNG.

Table 01 · Pilot results

PoolTogether pilot metrics

PublishedCase study1 run/modelManual GT
Detection, quality, runtime, token, and cost observations from one run per model.
MetricGPT-5.6 SolGrok 4.6 Build
Detection score100%0.1%
Recall100%50%
Precision66.7%25%
Wall time8m 26s26m 7s
Processed tokens813k3.95m
Estimated cost$1.32$3.24
Table 01Detection, quality, runtime, token, and cost observations from one run per model.

Harnesses differed. Ground truth was manually reconstructed; cost bases are not invoice-equivalent.

BMSbms.kyokasuigetsu.xyz
Static image: https://bms.kyokasuigetsu.xyz/tables/pilot-metrics.png

Table 02 · Reconstructed quality

Reference-based quality metrics

Active researchReconstructed scoring completeFrozen2 retained non-successesOfficial judge not run
Intention-to-treat reconstructed metrics across 36 scheduled runs and 129 reference opportunities per system.
Reconstructed metricGPT-5.6 Sol · HighGrok 4.6 Build · High
Reference detections52/12947/129
Micro recall40.31%36.43%
Macro audit recall46.06%41.81%
Award-weighted recall6.81%4.78%
Closed-world precision50.49%45.63%
Closed-world F144.83%40.52%
Table 02Intention-to-treat reconstructed metrics across 36 scheduled runs and 129 reference opportunities per system.

Codex-reconstructed manual scoring, not official API-backed EVMbench grading. Conventional accuracy is undefined.

BMSbms.kyokasuigetsu.xyz
Static image: https://bms.kyokasuigetsu.xyz/tables/campaign-quality.png

Table 03 · Finding dispositions

Reference matches and novel-valid findings

Active researchReconstructed scoring completeFrozen2 retained non-successesOfficial judge not run
Closed-world confusion counts beside the separate expert-validity review of findings outside the reference set.
Finding dispositionGPT-5.6 Sol · HighGrok 4.6 Build · High
TP / FP / FN, closed world52 / 51 / 7747 / 56 / 82
Novel-valid findings4951
Expert-validity precision98.06%96.12%
Conventional accuracyN/AN/A
Table 03Closed-world confusion counts beside the separate expert-validity review of findings outside the reference set.

An unmatched finding is not automatically invalid. Conventional accuracy is undefined for this open-ended audit task.

BMSbms.kyokasuigetsu.xyz
Static image: https://bms.kyokasuigetsu.xyz/tables/campaign-findings.png

Table 04 · Paired inference

Reconstructed effects and uncertainty

Active researchReconstructed scoring completeFrozen2 retained non-successesOfficial judge not run
GPT-minus-Grok paired effects across 12 audit clusters with hierarchical-bootstrap intervals and exact audit-level sign flips.
Paired reconstructed endpointGPT minus Grok95% CIExact p
Award-weighted recall, primary8.83 pp-6.88 to 25.48 pp0.1484
Micro reference recall4.26 pp-5.32 to 14.35 pp0.3359
Closed-world precision6.95 pp-11.17 to 21.94 pp0.4150
Closed-world F17.01 pp-2.71 to 16.30 pp0.1055
Table 04GPT-minus-Grok paired effects across 12 audit clusters with hierarchical-bootstrap intervals and exact audit-level sign flips.

The primary interval crosses zero. These are reconstructed manual estimates, not official EVMbench results.

BMSbms.kyokasuigetsu.xyz
Static image: https://bms.kyokasuigetsu.xyz/tables/campaign-inference.png

Table 05 · Frozen protocol

Controlled campaign design

Active researchReconstructed scoring completeFrozen2 retained non-successesOfficial judge not run
Frozen dimensions of the paired 12-audit, three-replicate comparison.
DimensionFrozen design
SystemsGPT-5.6 Sol · High via Codex CLI; Grok 4.6 Build · High via Grok CLI
Sample12 audits × 3 replicates × 2 systems = 72 assignments
User inputByte-identical prompt and deterministic source bundle within each paired block
AccessNo task tools, shell, web, browser, plugins, skills, memory, workflows, or subagents
RuntimeFresh workspace and session; sequential runs on the same Apple Silicon host
Timeout1,800 seconds from launch to final response
OutputOne final Markdown audit report captured by the harness
Table 05Frozen dimensions of the paired 12-audit, three-replicate comparison.

Provider system prompts, tokenizers, CLIs, compaction, safety routing, and cloud backends remain system-level differences.

BMSbms.kyokasuigetsu.xyz
Static image: https://bms.kyokasuigetsu.xyz/tables/campaign-design.png

Table 06 · Operational summary

Runtime, tokens, and recorded cost

Active researchReconstructed scoring completeFrozen2 retained non-successesOfficial judge not run
Operational aggregates across 35 complete reports per system.
SystemMedian wallMean tokensTotal tokensMean costTotal cost
GPT-5.6 Sol · High4m 24s99.87k3.5m$0.75 (API-equivalent)$26.08
Grok 4.6 Build · High10m 25s124.16k4.35m$0.41 (provider-reported)$14.31
Table 06Operational aggregates across 35 complete reports per system.

Grok cost is provider-reported; GPT-5.6 Sol cost is API-equivalent. The two price bases are not invoice-equivalent.

BMSbms.kyokasuigetsu.xyz
Static image: https://bms.kyokasuigetsu.xyz/tables/campaign-operations.png

Table 07 · Frozen integrity

Collection and pairing checks

Active researchReconstructed scoring completeFrozen2 retained non-successesOfficial judge not run
Integrity state of the frozen 72-assignment collection before reconstructed ground truth was opened.
Integrity checkState
Terminal assignments72 of 72
Frozen reports70 complete reports plus 2 retained non-success outcomes
Paired blocks36 complete; 0 unpaired
Ground truth opened during collectionNo
PoolTogether pilot includedNo
Table 07Integrity state of the frozen 72-assignment collection before reconstructed ground truth was opened.

Both non-successes remain in the frozen ledger and intention-to-treat denominators.

BMSbms.kyokasuigetsu.xyz
Static image: https://bms.kyokasuigetsu.xyz/tables/campaign-integrity.png

Table 09 · Verified-proof outcomes

Verified proofs per task and condition

Local · fully offlineFeasibility pilot6 replicates / cell0/54 runs terminalQuality pending
Verified proofs out of 6 replicates in each curated task × condition cell. A run counts as verified only when the submitted test compiles on both revisions, fails on the vulnerable revision, and passes on the fixed revision.
Task (family)Tool-assistedStatic (no tools)Budgeted triage
share-inflation (asset/share rounding)pendingpendingpending
reentrancy-withdraw (external-call ordering)pendingpendingpending
uninitialized-treasury (initialization authorization)pendingpendingpending
Table 09Verified proofs out of 6 replicates in each curated task × condition cell. A run counts as verified only when the submitted test compiles on both revisions, fails on the vulnerable revision, and passes on the fixed revision.

Local feasibility study, 6 replicates per cell; not an official EVMbench score. Cells with no terminal run read "pending", never zero.

BMSbms.kyokasuigetsu.xyz
Static image: https://bms.kyokasuigetsu.xyz/tables/verify-outcomes.png

Table 10 · Verified-proof rate

Verified-proof rate by condition

Local · fully offlineFeasibility pilot6 replicates / cell0/54 runs terminalQuality pending
Verified-proof rate over terminal runs per condition (6 replicates × 3 tasks), with a hierarchical bootstrap 95% CI resampling task clusters then replicates.
ConditionAgent accessTerminalVerifiedRate95% CI
Tool-assistedBounded search/read + forge build/test feedback, then one submitted proof0pendingpendingpending
Static (no tools)Full source context (target + provided test harness); no tool feedback0pendingpendingpending
Budgeted triageA strict budget of four inspect operations, then one submission0pendingpendingpending
Table 10Verified-proof rate over terminal runs per condition (6 replicates × 3 tasks), with a hierarchical bootstrap 95% CI resampling task clusters then replicates.

The CI clusters on three tasks, so intervals are wide by construction. Quality is pending until a condition is fully terminal; pending is never shown as zero. This is an agent-system study, not a model leaderboard.

BMSbms.kyokasuigetsu.xyz
Static image: https://bms.kyokasuigetsu.xyz/tables/verify-conditions.png

Table 11 · Frozen protocol

Conditions and budgets

Local · fully offlineFeasibility pilot6 replicates / cell0/54 runs terminalQuality pending
The three frozen conditions run over the same three tasks, with their turn/tool budgets and wall-time caps.
ConditionAgent accessTurn / tool budgetWall cap
Tool-assistedBounded search/read + forge build/test feedback, then one submitted proof12 turns / 10 tools1200 s
Static (no tools)Full source context (target + provided test harness); no tool feedback3 turns / 0 tools900 s
Budgeted triageA strict budget of four inspect operations, then one submission6 turns / 4 tools900 s
Table 11The three frozen conditions run over the same three tasks, with their turn/tool budgets and wall-time caps.

The agent receives only the vulnerable workspace; the fixed revision is never mounted. Foundry ffi and filesystem permissions are disabled in every task.

BMSbms.kyokasuigetsu.xyz
Static image: https://bms.kyokasuigetsu.xyz/tables/verify-protocol.png

Table 12 · Model & runtime lock

Frozen local model and runtime

Local · fully offlineFeasibility pilot6 replicates / cell0/54 runs terminalQuality pending
The exact locally served model and runtime configuration, recorded in runtime.lock.json before any run.
FieldValue
Requested specQwen/Qwen3.8-27B-FP8
Served locallyqwen3.8:27b → alias qwen3.8-27b-local
QuantizationQ4_K_M (requested FP8)
Architectureqwen35 · 27.32B params
Context length262144 native · served at 8192
Runtime engineOllama 0.32.14, loopback 127.0.0.1:11435, cloud disabled
HostApple M4 Pro · 24 GB unified memory · arm64
Samplingthink=false, temperature=1, top_p=0.95, num_predict=4096
Replicates6 per cell · seed = 1000 + replicate
Harness versionv2 (forge-std-subset test base)
Manifest SHA-256ec9bb426e93b2300
Table 12The exact locally served model and runtime configuration, recorded in runtime.lock.json before any run.

The requested FP8 spec and native 262144-token context do not fit 24 GB; the same model is served at Q4_K_M with an 8192-token context. This is a documented adaptation, not a different model and not a cloud fallback. No native long-context performance is claimed.

BMSbms.kyokasuigetsu.xyz
Static image: https://bms.kyokasuigetsu.xyz/tables/verify-model.png

Table 13 · Operational telemetry

Wall time, generated tokens, and localisation

Local · fully offlineFeasibility pilot6 replicates / cell0/54 runs terminalQuality pending
Operational aggregates across all terminal runs. These are operational observations only and do not measure finding quality.
MetricValue
Median wall time / runpending
Total collection wall timepending
Generated tokens (total)pending
Generated tokens (median / run)pending
Tool calls (total)pending
Source localised (tool-enabled runs)pending
Table 13Operational aggregates across all terminal runs. These are operational observations only and do not measure finding quality.

Operational telemetry is kept separate from the verified-proof endpoint. Wall time and generated tokens do not prove correctness. Pending cells are never shown as zero.

BMSbms.kyokasuigetsu.xyz
Static image: https://bms.kyokasuigetsu.xyz/tables/verify-operational.png

Table 14 · GLM-5.3 verified proofs

GLM-5.3 (ZCode) verified proofs per task and condition

Agent system · GLM-5.3 + ZCode CLIn = 1 run / cellOperator contamination disclosedDeterministic evaluatorNot a model leaderboard
Verified proofs out of 1 run in each curated task × condition cell for the GLM-5.3 (ZCode CLI) agent system. A run counts as verified only when the submitted test compiles on both revisions, fails on the vulnerable revision, and passes on the fixed revision.
Task (family)Tool-assistedStatic (no tools)Budgeted triage
share-inflation (asset/share rounding)1/11/11/1
reentrancy-withdraw (external-call ordering)1/11/11/1
uninitialized-treasury (initialization authorization)1/11/11/1
Table 14Verified proofs out of 1 run in each curated task × condition cell for the GLM-5.3 (ZCode CLI) agent system. A run counts as verified only when the submitted test compiles on both revisions, fails on the vulnerable revision, and passes on the fixed revision.

Agent-system arm (GLM-5.3 + ZCode harness), n = 1 run per cell, same fixtures and frozen evaluator as the local pilot; operator-role contamination is disclosed in the report. Cells with no terminal run read "pending", never zero. Not an official EVMbench score.

BMSbms.kyokasuigetsu.xyz
Static image: https://bms.kyokasuigetsu.xyz/tables/glm-outcomes.png

Table 15 · GLM-5.3 verified-proof rate

GLM-5.3 (ZCode) verified-proof rate by condition

Agent system · GLM-5.3 + ZCode CLIn = 1 run / cellOperator contamination disclosedDeterministic evaluatorNot a model leaderboard
Verified-proof rate over terminal runs per condition (1 run × 3 tasks) for the GLM-5.3 agent system, with the parent pilot's hierarchical bootstrap 95% CI.
ConditionAgent accessTerminalVerifiedRate95% CI
Tool-assistedBounded search/read + forge build/test feedback, then one submitted proof33/3100%100–100%
Static (no tools)Full source context (target + provided test harness); no tool feedback33/3100%100–100%
Budgeted triageA strict budget of four inspect operations, then one submission33/3100%100–100%
Table 15Verified-proof rate over terminal runs per condition (1 run × 3 tasks) for the GLM-5.3 agent system, with the parent pilot's hierarchical bootstrap 95% CI.

The interval is degenerate at 9/9 and n is 1 per cell: this is a descriptive result only, with no inferential claim. Condition-level contrasts are confounded by within-session carryover (tools ran before static and triage).

BMSbms.kyokasuigetsu.xyz
Static image: https://bms.kyokasuigetsu.xyz/tables/glm-conditions.png

Table 16 · Frozen arm protocol

GLM-5.3 arm conditions and budgets

Agent system · GLM-5.3 + ZCode CLIn = 1 run / cellOperator contamination disclosedDeterministic evaluatorNot a model leaderboard
The three frozen conditions of the parent pilot, reused unchanged (same turns, tools, and wall caps) over the same three tasks.
ConditionAgent accessTurn / tool budgetWall cap
Tool-assistedBounded search/read + forge build/test feedback, then one submitted proof12 turns / 10 tools1500 s
Static (no tools)Full source context (target + provided test harness); no tool feedback3 turns / 0 tools1200 s
Budgeted triageA strict budget of four inspect operations, then one submission6 turns / 4 tools1200 s
Table 16The three frozen conditions of the parent pilot, reused unchanged (same turns, tools, and wall caps) over the same three tasks.

Wall time runs begin→final and includes GLM-5.3 + ZCode inference between turns. The agent interacts with the task workspace only through the driver, which enforces the parent action policy; the fixed revision is never mounted.

BMSbms.kyokasuigetsu.xyz
Static image: https://bms.kyokasuigetsu.xyz/tables/glm-protocol.png

Table 17 · Agent-system lock

Frozen GLM-5.3 + ZCode agent-system lock

Agent system · GLM-5.3 + ZCode CLIn = 1 run / cellOperator contamination disclosedDeterministic evaluatorNot a model leaderboard
The agent system under test, recorded in agent.lock.json before any run and hashed into every run ledger.
FieldValue
Model identifierbuiltin:zai-start-plan/GLM-5.3
Runtime engineZCode CLI/desktop harness (one stateful session)
ZCode app3.8.1 · CLI bundle sha256 9318f60fb8c2c3bc…
Providerzai (OAuth), Anthropic-compatible endpoint
Reported limitscontext 1,000,000 · output 128,000
Inference controlnone (no seed, no sampler control)
Token meteringnot available — recorded n/a, never zero
SandboxDocker --network none · cap-drop ALL · no-new-privileges · solc 0.8.35
HostApple M4 Pro · 24 GB · arm64 · macOS 27.0
Lock frozenbefore first run · 2026-08-21T19:59:41Z
Replicates1 per cell (deviation from parent 6/cell justified pre-run)
Table 17The agent system under test, recorded in agent.lock.json before any run and hashed into every run ledger.

Inference is cloud-served through the ZCode runtime: sampler settings are not operator-controlled and token telemetry does not exist (recorded n/a, never imputed). The arm deviates from the parent 6-replicate plan: 1 run per cell, justified before collection because a single stateful session cannot produce independent replicates.

BMSbms.kyokasuigetsu.xyz
Static image: https://bms.kyokasuigetsu.xyz/tables/glm-agent-lock.png

Table 18 · Operational telemetry

GLM-5.3 arm wall time, tools, and localisation

Agent system · GLM-5.3 + ZCode CLIn = 1 run / cellOperator contamination disclosedDeterministic evaluatorNot a model leaderboard
Operational aggregates across all terminal runs of the GLM-5.3 arm. These are operational observations only and do not measure finding quality.
MetricValue
Median wall time / run0m 09s
Total collection wall time5m 11s
Generated tokens (total)n/a (no token metering)
Tool calls (total)15
Source localised (tool-enabled runs)6/6
Table 18Operational aggregates across all terminal runs of the GLM-5.3 arm. These are operational observations only and do not measure finding quality.

Wall time includes GLM-5.3 + ZCode inference between turns and is not comparable to the parent pilot's automated local loop. Token telemetry does not exist for this runtime and is shown as n/a, never zero.

BMSbms.kyokasuigetsu.xyz
Static image: https://bms.kyokasuigetsu.xyz/tables/glm-operational.png

Table 19 · Adjacent arms

Local verify arms side by side (never pooled)

Agent system · GLM-5.3 + ZCode CLIn = 1 run / cellOperator contamination disclosedDeterministic evaluatorNot a model leaderboard
Adjacent descriptive results across arms that share fixtures or lineage. Denominators are per-arm; protocols, harness generations, and endpoints differ.
Arm (agent system)HarnessRunsVerifiedMedian wallPrimary caveat
GLM-5.3 (ZCode CLI)v2 fairer harness99/90m 09sOperator-role contamination disclosed; n = 1/cell; no token metering
Qwen3.8-27B · Q4_K_M (local Ollama)v1 strict harness90/96m 48s8K-context Q4_K_M local serving, thinking disabled; strict v1 prompt
Qwen3.8-27B · Q4_K_M (local Ollama) (v2)v2 fairer harness1 of 54 scheduledpendingpendingCollection incomplete (2 partial run dirs); quality withheld, never zero
Table 19Adjacent descriptive results across arms that share fixtures or lineage. Denominators are per-arm; protocols, harness generations, and endpoints differ.

These are not comparable measurements and must not be read as a leaderboard. The Qwen v2 arm (same harness generation as the GLM arm) is still collecting: its quality cells read "pending" and are never shown as zero. The controlled Grok/GPT campaign measures a different endpoint (reported findings on audits) and is excluded from this table by design.

BMSbms.kyokasuigetsu.xyz
Static image: https://bms.kyokasuigetsu.xyz/tables/glm-cross-arm.png

Table 20 · Adjacent cross-system

EVMBench Detect recall by agent system (never pooled)

Agent system · GLM-5.3 + ZCode CLIDetect mode · 12 audits, 1 run/auditSingle-reviewer self-scoredReconstructed manual scoringNot official EVMbench grading
Reference-detection recall on the 12 frozen audits. GLM-5.3 (ZCode) ran 1 run/audit scored by itself; GPT-5.6 Sol and Grok 4.6 Build ran 3 runs/audit with two independent reviewers plus adjudication.
Agent systemRunsRefs detectedMicro recallMacro recallAward-weightedPrecision (closed)F1 (closed)Median wall
GLM-5.3 · ZCode CLI 3.8.1 (self-scored)125/4311.6%12.1%1.91%29.4%16.7%1m 09s
GPT-5.6 Sol · High (Codex CLI)3652/12940.3%46.1%6.81%50.5%44.8%4m 24s
Grok 4.6 Build · High (Grok CLI)3647/12936.4%41.8%4.78%45.6%40.5%10m 25s
Table 20Reference-detection recall on the 12 frozen audits. GLM-5.3 (ZCode) ran 1 run/audit scored by itself; GPT-5.6 Sol and Grok 4.6 Build ran 3 runs/audit with two independent reviewers plus adjudication.

Adjacent descriptive results under different protocols: replicate counts (1 vs 3), scorer independence, harness CLI, and session structure all differ. The GLM column is a single-reviewer self-scored reconstruction and is not official EVMbench grading. Never pooled into one leaderboard.

BMSbms.kyokasuigetsu.xyz
Static image: https://bms.kyokasuigetsu.xyz/tables/glm-detect-quality.png

Table 21 · GLM arm outcomes

GLM-5.3 (ZCode) references detected per audit

Agent system · GLM-5.3 + ZCode CLIDetect mode · 12 audits, 1 run/auditSingle-reviewer self-scoredReconstructed manual scoringNot official EVMbench grading
Per-audit reference detection and finding counts for the GLM-5.3 (ZCode CLI) arm: one run per audit against the frozen bundles, scored by the same session that wrote the reports.
AuditRefs detectedFindings reportedTPFP (closed)FNWall
NextGen (NFT minter)0/240423m 04s
Ethereum Credit Guild (lending)1/221113m 13s
Curves (social tokens)3/423011m 01s
reNFT (rental escrow)0/620262m 06s
Taiko (rollup)1/521142m 29s
Size (credit market)0/410141m 09s
Basin (AMM)0/200021m 01s
Phi (creator creds)0/610160m 53s
Wildcat (credit vault)0/100010m 33s
Forte (float math)0/500050m 39s
Virtuals (agent protocol)0/410141m 45s
Panoptic (options vault)0/210120m 40s
Table 21Per-audit reference detection and finding counts for the GLM-5.3 (ZCode CLI) arm: one run per audit against the frozen bundles, scored by the same session that wrote the reports.

Single-reviewer self-scored reconstruction; not official EVMbench grading. Reported findings that matched no reference are closed-world false positives; their expert validity was not adjudicated.

BMSbms.kyokasuigetsu.xyz
Static image: https://bms.kyokasuigetsu.xyz/tables/glm-detect-runs.png

Table 22 · Adjacent cross-system

EVMBench Detect recall by agent system, four arms (never pooled)

Agent system · Ox Alpha + OpenCode 1.18.20Detect mode · 12 audits, 1 run/auditClean-room re-run (v3)Single-reviewer self-scoredNot official EVMbench grading
Reference-detection recall on the same 12 frozen audits. Ox Alpha (OpenCode) and GLM-5.3 (ZCode) ran 1 run/audit scored by themselves; GPT-5.6 Sol and Grok 4.6 Build ran 3 runs/audit with two independent reviewers plus adjudication.
Agent systemRunsRefs detectedMicro recallMacro recallAward-weightedPrecision (closed)F1 (closed)Median wall
Ox Alpha · OpenCode 1.18.20 (self-scored)124/439.3%7.6%1.03%25.0%13.6%2m 50s
GPT-5.6 Sol · High (Codex CLI)3652/12940.3%46.1%6.81%50.5%44.8%4m 24s
Grok 4.6 Build · High (Grok CLI)3647/12936.4%41.8%4.78%45.6%40.5%10m 25s
GLM-5.3 · ZCode CLI 3.8.1 (self-scored)125/4311.6%12.1%1.91%29.4%16.7%1m 09s
Table 22Reference-detection recall on the same 12 frozen audits. Ox Alpha (OpenCode) and GLM-5.3 (ZCode) ran 1 run/audit scored by themselves; GPT-5.6 Sol and Grok 4.6 Build ran 3 runs/audit with two independent reviewers plus adjudication.

Adjacent descriptive results under different protocols: replicate counts (1 vs 3), scorer independence, harness CLI, and session structure all differ. Ox and GLM columns are single-reviewer self-scored reconstructions, not official EVMbench grading. The Ox figure is the clean-room v3 run (pre-freeze blindness held); two earlier answer-key-contaminated attempts were discarded and archived. Never pooled into one leaderboard.

BMSbms.kyokasuigetsu.xyz
Static image: https://bms.kyokasuigetsu.xyz/tables/oxalpha-detect-quality.png

Table 23 · Ox Alpha arm outcomes

Ox Alpha (OpenCode) references detected per audit

Agent system · Ox Alpha + OpenCode 1.18.20Detect mode · 12 audits, 1 run/auditClean-room re-run (v3)Single-reviewer self-scoredNot official EVMbench grading
Per-audit reference detection and finding counts for the Ox Alpha (OpenCode) arm: one run per audit against the frozen bundles, scored by the same session that wrote the reports.
AuditRefs detectedFindings reportedTPFP (closed)FNWall
NextGen (NFT minter)0/270724m 23s
Ethereum Credit Guild (lending)0/2202210m 41s
Curves (social tokens)3/433112m 50s
reNFT (rental escrow)0/600065m 40s
Taiko (rollup)0/5000518m 10s
Size (credit market)0/400041m 34s
Basin (AMM)0/200021m 13s
Phi (creator creds)1/611051m 22s
Wildcat (credit vault)0/100010m 58s
Forte (float math)0/500050m 49s
Virtuals (agent protocol)0/420245m 59s
Panoptic (options vault)0/200021m 39s
Table 23Per-audit reference detection and finding counts for the Ox Alpha (OpenCode) arm: one run per audit against the frozen bundles, scored by the same session that wrote the reports.

Single-reviewer self-scored reconstruction from the clean-room v3 run (pre-freeze blindness held); not official EVMbench grading. Reported findings that matched no reference are closed-world false positives; their expert validity was not adjudicated.

BMSbms.kyokasuigetsu.xyz
Static image: https://bms.kyokasuigetsu.xyz/tables/oxalpha-detect-runs.png

Table 08 · Publication workflow

What the benchmark fixes and records

The four-stage workflow used to make benchmark runs reproducible and publication claims auditable.
StageWhat is fixedWhat is recorded
PrepareCommit, scope, prompt, controlsHashes and environment
RunFresh workspace, tool policyTime, tokens, tool events
GradeRubric and ground truthMatches, misses, disputes
PublishDisclosure schemaResults and limitations
Table 08The four-stage workflow used to make benchmark runs reproducible and publication claims auditable.

Controls and disclosure fields vary by study; each report documents deviations and unresolved system-level differences.

BMSbms.kyokasuigetsu.xyz
Static image: https://bms.kyokasuigetsu.xyz/tables/methodology-workflow.png