Sources

Where every speed on this site comes from, how we filter it, and what we leave out.

llm-speed

Signed community runs of a fixed benchmark suite on current models, with runtime, version, quant and token counts. We use the chat-short workload and take the median per model and card.

Licence: Data from llm-speed (https://llm-speed.com/data), CC BY 4.0, median computed by us.

Results on this site: 16

llama.cpp

llama-bench results posted in llama.cpp's own benchmark threads (Apple Silicon #4167, CUDA #15013, Vulkan #10879) and the DGX Spark page in the repo. Same model, quant and command for everyone; each comment has its own link.

Licence: Discussion posts carry no licence; the repo, including the DGX Spark page, is MIT. We quote single figures with a link.

Results on this site: 14

Apple Silicon thread · CUDA thread · Vulkan thread · DGX Spark page

oMLX

Community runs submitted from the oMLX Mac app: chip, memory, model, quant and context for each run. We use 4-bit runs at 1k context and take the median run per chip and model.

Licence: The app is Apache-2.0; no licence is stated for the results. We quote single figures with a link.

Results on this site: 14

Strix Halo Guide

llama-bench runs on a Beelink GTR9 Pro (Ryzen AI Max+ 395) with build, kernel, driver, repeats and raw logs for each row.

Licence: MIT (GitHub repository).

Results on this site: 7

GitHub data

NetworkChuck

Controlled mlx-lm runs on a Mac Studio M5 Ultra 80-core, 256GB: 512-token prompt, median of 5 runs, raw JSON.

Licence: MIT.

Results on this site: 2

LocalScore

A fixed test set run with llamafile 0.9.2 on small models. The figure is an average over its tests, including long outputs, so it reads lower than llama-bench. Used only where a machine has no other result for that model, and never in the llama-bench tables.

Licence: The code is Apache-2.0; no licence is stated for the results. We quote single figures with a link.

Results on this site: 2

stared

llama.cpp runs on an M5 Max with the commands in the repository.

Licence: No licence stated. We quote single figures with a link.

Results on this site: 1

Our filters

  • No quant stated: hidden.
  • Runs more than 2x apart: hidden.
  • A single run is labelled "1 run".
  • One engine per result. llm-speed medians use llama.cpp runs where they exist, otherwise the one engine that was used, named next to the figure.
  • oMLX: 4-bit runs only, so chips are compared on the same quant.
  • LocalScore: only where a machine has no other result for that model, labelled "LocalScore (llamafile, averaged)", never in the llama-bench tables.
  • Several results for one model and device: we show one, by source priority: llama.cpp's fixed test first; then llm-speed, oMLX, Strix Halo Guide, NetworkChuck and stared; then aidatatools and Reddit; LocalScore last. Within a tier, the result with more runs wins, then the newest.
  • Every figure links to the run, post or file it came from. No estimates; a missing figure means we found none.

Engines, quants and context lengths still differ between sources. Each figure names its quant and engine.

Results we found but don't show (16)
  • Gemma 4 12B on M5 Max 40-core: 48.2 tok/s (Reddit). Reddit post; replaced by an oMLX 4-bit run with a published method.
  • Gemma 4 26B-A4B on M5 Max 40-core: 96.3 tok/s (Reddit). Reddit post; replaced by an oMLX 4-bit run with a published method.
  • gpt-oss-120b on M5 Max 40-core: 79.1 tok/s (bestllmfor). Aggregator page, no published method or raw data; replaced by an oMLX result.
  • gpt-oss-120b on M5 Ultra 80-core: 130 tok/s (youtube). Stated in a video, no checkable data; oMLX runs for this chip don't state a quant.
  • gpt-oss-20b on RTX 5090 32GB: 193.9 tok/s (llm-speed). 2 run(s), none states a quant or prompt size; runs range 69–318.
  • Llama 3.1 8B on RTX 5090 32GB: 230.8 tok/s (llm-speed). 2 run(s), none states a quant or prompt size; runs range 229–232.
  • Qwen3 32B on RTX 5090 32GB: 68 tok/s (llm-speed). 2 run(s), none states a quant or prompt size; runs range 67–69.
  • Qwen3-Coder 30B-A3B on RTX 5090 32GB: 164.2 tok/s (llm-speed). 2 run(s), none states a quant or prompt size; runs range 68–260.
  • Qwen3.6 35B-A3B on RTX 5090 32GB: 220.4 tok/s (llm-speed). 26 run(s), none states a quant or prompt size; runs range 209–224.
  • Qwen3.6 27B on RTX 5090 32GB: 70.6 tok/s (llm-speed). 74 run(s), none states a quant or prompt size; runs range 40–74.
  • Qwen3-Coder 30B-A3B on M5 Ultra 64-core: 99.8 tok/s (oMLX). Only 8-bit runs; other chips use 4-bit.
  • gpt-oss-20b on M5 Pro 20-core: 64.6 tok/s (oMLX). All 4 runs from one submission on one day, about 25% below the 16-core M5 Pro on the same quant; decode speed is limited by memory bandwidth, which both chips share.
  • Qwen3.6 35B-A3B on M5 Ultra 64-core: 123.7 tok/s (oMLX). Only 8-bit runs; other chips use 4-bit.
  • Qwen3.8 27B on M5 Ultra 64-core: 30.5 tok/s (oMLX). Only 8-bit runs; other chips use 4-bit.
  • Qwen3.8 27B on M5 Ultra 80-core: 32.4 tok/s (oMLX). Only 8-bit runs; other chips use 4-bit.
  • Gemma 4 26B-A4B on M5 Pro 20-core: 50.8 tok/s (oMLX). Only 8-bit runs; other chips use 4-bit.

Spot a wrong number? [email protected]