HarnessTax priced 21 model-harness pairs. The harness moved the bill further than the score.

HarnessTax, generated 2026-09-16 16:23 UTC, ran seven models through three harnesses — Claude Code, Codex CLI and Pi — on SWE-bench Lite and Terminal-Bench 2.0: 21 pairs, 30 unique tasks, three repetitions each.¹ On SWE-bench Lite each harness is compared against Pi. Of the fourteen comparisons, one survives Holm correction on success rate: Claude Opus 4.8 under Codex, +6.7 points over Pi, 95% CI [2.2, 11.1], p=0.002, Holm-adjusted 0.042.²

Claude Fable 5 is the case to open. Under Claude Code it scores 97.8% at $1.3293 per rollout; under Pi, 96.7% at $0.6657.³ The rate difference is +1.1 points, CI [-3.3, +6.7], p=0.764: one task went to Claude Code, one to Pi, 28 landed identically.² Same model, same thirty tasks, roughly double the cost per rollout, on a sample of thirty tasks.