Breaking

Google Releases Encoder-Free Gemma 4 12B Model

June 15, 2026 at 17:17 EDT

Foundation Models
Open Source
AI Agents

Google DeepMind on June 3, 2026 released Gemma 4 12B, a unified multimodal model that drops dedicated image and audio encoders. It runs on laptops with around 16GB of memory and is available immediately under Apache 2.0.

June 3, 2026 · Google DeepMind

Gemma 4 12B Drops the Encoders

An open-weight multimodal model that feeds raw image patches and audio waveforms straight into the LLM — no separate vision or audio encoders — built to run on a 16GB consumer laptop.

11.95B

dense parameters

256K

token context length

16GB

RAM to run locally

30+

tokens/sec (Q4 GGUF)

Stripping out the encoders

Parameters spent on dedicated encoders — replaced by one lightweight projection layer.

~550M

old vision encoder
(larger models)

~450M

E2B/E4B vision
+ audio encoders

→

~35M

linear projection
layer (Gemma 4 12B)

Raw 48×48 image patches and 40ms waveforms go straight into the LLM embedding space.

Benchmark scores

77.2%

MMLU Pro

78.8%

GPQA Diamond

72.0%

LiveCodeBench v6

69.1%

MMMU Pro (vision)

What developers like

Agentic workflows & audio transcription practical at 16GB
Runs locally with lower latency than an API
Strong on coding and document understanding

The caveats

More work shifts onto the LLM itself
May miss very fine-grained visual features
Calls for more published comparisons

Google claims Gemma 4 12B approaches the 26B A4B MoE in performance while using less than half the memory — shipped as open weights under Apache 2.0, with pretrained, instruction-tuned, and quantization-aware (QAT) versions.

Continue reading

The rest of this article is for AI News Blitz readers. Choose an option below to keep reading.