Breaking

Six AI Agent Harnesses Compared as Tests Show 20-Point Gaps From Runtime Alone

September 3, 2026 at 06:47 EDT

AI Agents
Software Dev & Coding
Open Source

A 26-minute practitioner walkthrough published this week maps six AI agent harnesses — Oh My Pi, Pi, Claude Code, ZCode, Codex, and Warp — as independent tests show the same language model can swing by 20 percentage points depending on the runtime wrapped around it. The comparison captures a shift that has defined coding-agent work in 2026: builders are choosing the loop, tools, context manager, and permissions around a model, not only the model itself.

September 2026 · Agent harnesses

Six runtimes now sit between you and the model

The same model can swing 20 points on pass rate depending on the harness. Developers are comparing Oh My Pi, Pi, Claude Code, ZCode, Codex, and Warp as a working stack.

Harnesses in the circulating stack

88%

Oh My Pi pass rate (Kimi K3, 25 tasks)

68%

Codex pass rate on the same test

3×

Less context per turn with Pi vs heavier harnesses

Same model, different pass rate

Composio held Kimi K3 fixed across harnesses. Column height is proportional to observed pass rate (9 blocks vs 7).

88%

Oh My Pi

68%

Codex

20-point gap with the model, tools, and tasks held constant.

Context sent per turn

Databricks: Pi sent about 3× less context per turn than Claude Code or Codex at equal quality.

1×

Pi (lean)

3×

Heavier harness

Cost per task moved by more than 2× at the same quality.

The working stack

Oh My Pi

→

Claude Code

→

ZCode

→

Codex

→

Warp

Minimal core → native vendor loop → GLM desktop ADE → sandboxed unattended runs → multi-harness control plane.

What is working

Oh My Pi led reliability in the Kimi K3 tool-use test. Pi led speed and token efficiency. Matching the harness to the job — refactors, CI, local models — is now the explicit advice.

What still breaks

Vendor harnesses were not on native models in that test. Builders still hit lock-in, opinionated defaults, and the need to run several tools at once. No single product covers unattended depth, cheap context, and every model family.

Continue reading

The rest of this article is for AI News Blitz readers. Choose an option below to keep reading.