BMS

Start typing to search the publication.

EVMBench Detect · Cross-system comparison

EVMBench Detect — Ox Alpha vs GPT-5.6 Sol vs Grok 4.6 Build vs GLM-5.3

Reference-vulnerability detection on the same 12 frozen smart-contract audits: Ox Alpha (High), a free model served through an OpenCode session, vs the parent campaign's GPT-5.6 Sol and Grok 4.6 Build and the GLM-5.3 arm. Adjacent descriptive results under different protocols (1 vs 3 replicates; self-scored vs two independent reviewers). Ox additionally shared a session with an invalidated answer-key run, so its recall and precision are upper bounds. Reconstructed, not official EVMbench grading, and never pooled into a leaderboard.

Back to all figures

Reference-vulnerability detection on the same 12 frozen smart-contract audits: Ox Alpha (High), a free model served through an OpenCode session, vs the parent campaign's GPT-5.6 Sol and Grok 4.6 Build and the GLM-5.3 arm. Adjacent descriptive results under different protocols (1 vs 3 replicates; self-scored vs two independent reviewers). Ox additionally shared a session with an invalidated answer-key run, so its recall and precision are upper bounds. Reconstructed, not official EVMbench grading, and never pooled into a leaderboard.
Micro recall 9.3% (Ox, upper bound) vs 40.3% (Sol) vs 36.4% (Grok) vs 11.6% (GLM); median wall 170s vs 264s vs 625s vs 69s. Reconstructed manual scoring, not official EVMbench grading; Ox/GLM self-scored (zero reviewer independence); Ox shared a session with an invalidated answer-key run. Never pooled.
Comparison PNGComparison SVG 1200×675 social card Read the full Ox Alpha arm Read the GLM-5.3 arm