Breaking

Google Releases Encoder-Free Multimodal Model Gemma 4 12B

June 15, 2026 at 17:18 EDT

Foundation Models
Open Source
Media Generation

Google DeepMind on June 3, 2026 released Gemma 4 12B, a unified multimodal model with no dedicated encoders, as an open-weight model under the Apache 2.0 license. It processes text, images, audio and video natively and is promoted as running fully locally on a laptop-class machine with around 16GB of RAM.

June 3, 2026 · Google DeepMind

Gemma 4 12B drops the encoders — one Transformer, every modality

An open-weight (Apache 2.0) multimodal model that fully removes dedicated image and audio encoders, projecting raw pixels and waveforms straight into the LLM — and running locally on a 16GB laptop.

12B

Dense parameters, decoder-only

~16GB

Memory to run locally (Q4)

30+

Tokens/sec on Mac Mini / RTX 5060 Ti

Encoder-free: the parameters it deleted

Conventional models bolt on heavy encoders per modality. Gemma 4 12B replaces them with lightweight linear projection.

Image path

~550M

Old vision
encoder

~35M

New
embedder

Audio path

~305M

Old Conformer
encoder

Linear

40ms frame
projection

Takeaway: raw 48×48 image patches and 16kHz waveform frames feed directly into a single decoder-only Transformer.

Where 12B sits in the lineup

A mid-size dense model between edge variants and the larger models.

E2B / E4B

Edge · mobile / on-device

THIS

Dense 12B

Local / mid-size inference

26B A4B / 31B

Large · high-accuracy

Benchmarks — nearing the 26B model

GPQA Diamond78.8

LiveCode Bench72

Memory footprint — quantization

26.7GB

FP16

~6.7GB

~4× smaller — practical on a 16GB machine.

Continue reading

The rest of this article is for AI News Blitz readers. Choose an option below to keep reading.