Breaking

Artificial Analysis Launches AgentPerf, First Infrastructure Benchmark for AI Agents

June 12, 2026 at 17:15 EDT

AI Agents
Infra & Chips
Research & Papers

Around March 27, 2026, Artificial Analysis released AA-AgentPerf, the first open hardware benchmark built for AI agent workloads, and immediately opened submissions. Results and methodology are published on its methodology page and hardware benchmark.

March 27, 2026 · Artificial Analysis

AA-AgentPerf: the first hardware benchmark built for AI agent workloads

Using real coding-agent execution traces, it ranks accelerators by how many concurrent users they can serve — normalized per chip, per kW, per dollar and per rack.

200

turns per trajectory (load applied)

100K+

tokens of sequence length

hardware classes compared

NVIDIA's headline claim: 20× more agents per megawatt

Blackwell (B200) vs the prior Hopper generation, on long-context, multi-turn, tool-calling loads.

1×

Hopper

20×

Blackwell (B200)

Agents served per megawatt — power efficiency is the agent-era differentiator.

Why agent workloads break old token-per-second benchmarks

Model call

→

Tool use

→

Gather context

↺

One task chains dozens to hundreds of calls across multi-turn, long-context, parallel sessions — realities single-query metrics never captured.

Target models at launch

gpt-oss-120b · DeepSeek V3.2

Optimizations permitted

KV cache reuse · disaggregated prefill/decode · speculative decoding

Welcomed

Real workloads finally driving meaningful hardware benchmarks; gpt-oss-120b already running in production agentic deployments.

Caution

Concrete validation of pitfalls and limits is still scarce — the benchmark is brand new, with results published on a rolling basis.

Continue reading

The rest of this article is for AI News Blitz readers. Choose an option below to keep reading.