{"slug":"magnitudedev-magnitude","kind":"claim","status":"testing","tier":"claim","title":"Magnitude: is it really 2x faster than Ollama?","category":"local-models/runtimes","topic":"speed-claims","repos":[{"repo":"magnitudedev/magnitude","stars":7066,"license":"Apache-2.0"}],"claims":[{"claim":"An open-source local inference engine that tunes kernels to your hardware, over 2x faster than Ollama (2x faster decode than llama.cpp)","source":"https://x.com/Shadowfetch/status/2106312008434135485"}],"plan":"Install the Linux package in a container, start the server, download a recommended small model, and time decode tokens/second on a fixed prompt. Compare against the same task on Ollama on the same box.","who_its_for":"Anyone running local models who wants more speed for free.","explainer":"An inference engine that auto-tunes kernels per model and claims to beat Ollama by 2x. Rocky measures real decode speed on Rocky's GPU box. Note: a clean 2x verdict needs the exact same model on both engines, so Rocky is careful about which number means what.","measure":["Does it install and serve on Linux","Decode tokens/second on a small model","How that compares to Ollama on the same hardware"],"notes_so_far":["Method: pinned CLI 0.2.9, GPU job on Rocky's box, no paid APIs. Rocky will only call it 2x if the comparison is same-model, same-hardware.","GPU test queued; results land here when it runs."],"sneak_peek":["Smoke test: the .deb installed and the server started, but no model was picked (the catalog had no recommendations while the box's memory was busy), so the speed test returned HTTP 400.","Same-prompt baseline: Ollama ran qwen2.5-coder 7B at 45.3 tokens/s.","A claim rerun with the 120B model paused and 10 packet prompts is queued."],"charts":[],"gallery":[],"results":{"jobs":[{"id":26,"title":"magnitude: install, serve, pull a small model, measure decode tok/s for the 2x-faster-than-Ollama claim (GPU)","kind":"smoke","status":"done","started":"2026-10-18T05:46:35Z","finished":"2026-10-18T05:48:26Z","wall_seconds":111,"install_friction":null,"artifacts":["raw/magnitudedev-magnitude/job-26/logs/model.txt"]}],"smoke_only":true,"ingested":"2026-10-18T15:02:07Z"},"last_test_finished":"2026-10-18T05:48:26Z","results_in":true}