Google and Intel have detailed a setup that runs the lightweight Gemma 4 E2B model on the NPU of Intel Core Ultra processors. Via LiteRT's OpenVINO backend integration, they report roughly 1.3x faster prefill and 2.8x better performance-per-watt over GPU.
Google AI Edge × Intel · On-Device LLM
Gemma 4 now runs on the Core Ultra NPU — always-on AI at half the power
An OpenVINO backend for LiteRT lets the compact Gemma 4 E2B model offload inference to the AI Boost NPU in Core Ultra AI PCs — delivering faster prefill and far better performance-per-watt than the GPU path.
1.3×
Faster prefill vs GPU (WebGPU)
2.8×
Better performance-per-watt on NPU
<50%
Power draw of the GPU path
Performance-per-watt: NPU vs GPU
Each block = 1× efficiency unit (Core Ultra Series 3)
Low power and low heat enable an always-on local LLM — summarization or tool calls during a video call without thermal throttling or heavy battery drain.
Raw throughput by target
Gemma 4 E2B · 1024 prefill / 256 decode tokens · Lunar Lake
Time-to-first-token (GPU)
0.29s
WHAT WORKS
Memory ~1.5–2 GB vs 4–5 GB for llama.cpp-style tools
Reduced heat and fewer OS process kills
AOT + JIT compilation cuts startup latency
90+ classic ML models support NPU delegation
CAVEATS
NPU features shipped only in June — limited hands-on reports
Must reclaim memory when releasing a model
NPU driver setup effort required
Immature NPU support for large 26B / 31B models
The Open Question
How widely will dedicated-NPU power-efficiency gains be adopted for AI PC use cases built around offline operation and always-on background processing ? Gemma 4 (E2B/E4B) is a feature preview in OpenVINO 2026.0 / 2026.1 on Windows and Linux, with up to 32k context via mixed 2/4/8-bit quantization.
Continue reading The rest of this article is for AI News Blitz readers. Choose an option below to keep reading.
Already purchased? Sign in ✓ Signed in — this article isn’t included in your current plan.Unlocking the full article…