A newly circulated open-source inference tool called flashtensors promises to load large language models from SSD to GPU memory as much as 10x faster than conventional loaders, enabling cold starts of under two seconds and letting hundreds of models coexist on a single GPU—though its release has been shadowed by questions over whether it properly credits the project it was built upon.
Continue reading
The rest of this article is for AI News Blitz readers. Choose an option below to keep reading.
Already purchased? Sign in✓ Signed in — this article isn’t included in your current plan.
Unlocking the full article...
Contact us
Questions, corrections, or ideas? Send us a message here. We aim to reply by email within 24 hours.
Thank you. Your message has been sent. We aim to reply by email within 24 hours.Please enter your email address and a message.We could not send your message. Please try again in a little while.