Researchers from Meta, Google DeepMind, Cornell and NVIDIA have put a concrete number on how much a language model can memorize: roughly 3.6 bits per parameter, implying a 7-billion-parameter model can hold on the order of 3GB of training data verbatim. The finding comes from the paper "How much do language models memorize?" (arXiv:2505.24832), which was accepted to ICML 2026 and named an Outstanding Paper Honorable Mention (ICML listing).
Continue reading
The rest of this article is for AI News Blitz readers. Choose an option below to keep reading.
Already purchased? Sign in✓ Signed in — this article isn’t included in your current plan.
Unlocking the full article...
Contact us
Questions, corrections, or ideas? Send us a message here. We aim to reply by email within 24 hours.
Thank you. Your message has been sent. We aim to reply by email within 24 hours.Please enter your email address and a message.We could not send your message. Please try again in a little while.