Breaking

Cognition unveils FrontierCode; top model scores just 13.4%

June 9, 2026 at 10:42 EDT

Research & Papers
Software Dev & Coding
Foundation Models

Cognition, the maker of Devin, on June 8, 2026 introduced FrontierCode, a new coding benchmark that measures whether AI-generated code is actually mergeable into a real production codebase. On the hardest Diamond subset, even the leading Claude Opus 4.8 scored only 13.4%, with other major models posting even lower numbers. FrontierCode

June 8, 2026 · Cognition

FrontierCode: the bar isn't passing tests — it's getting your code merged

A new benchmark grades AI code the way a real maintainer would — quality, conventions, mergeability. On its hardest "Diamond" set, the best model in the world scores just 13.4%.

13.4%

Best model on the Diamond subset (Claude Opus 4.8)

81%

Lower false-positive rate vs SWE-Bench Pro

40h+

Authoring time per task, by 20+ real maintainers

Diamond subset · best of 5 runs

Even the leaders barely clear the bar

13.4%

6.3%

4.7%

3.8%

Claude Opus 4.8

GPT-5.5

Gemini 3.1 Pro

Kimi K2.6 (open)

Claude Opus 4.8 · same model, different difficulty

Scores collapse as the tasks get harder

51.8%

Extended
150 tasks

34.3%

Main
100 hardest

13.4%

Diamond
50 hardest

Why "passing tests" wasn't enough

SWE-Bench rewards code that works but is hard to maintain. FrontierCode grades on a weighted rubric and asks the only question that matters: would a maintainer merge this?

Behavioral correctness Regression safety Mechanical cleanliness Test correctness Scope Code quality

Welcomed

A realistic standard of "code that gets merged" that fixes SWE-Bench's blind spots — with real maintainers encoding human, subjective quality.

Cautioned

A useful reality check that coding isn't "solved" — but public data may saturate fast, with open questions on reproducibility and variance.

Continue reading

The rest of this article is for AI News Blitz readers. Choose an option below to keep reading.