Magnitude Says Hardware-Tuned Local Inference Can Outpace Ollama

Magnitude, an open-source inference engine for running open models locally, says it can deliver more than twice Ollama’s decode speed on average by tuning kernels for each model and device. Its published comparison covers 11 open models on Apple and NVIDIA systems, though results vary by workload and some individual tests favor Ollama.