Best GPU for local LLMs
Our pick for a graphics card for local LLMs is the NVIDIA RTX 3090 24GB (used): $700 / ≈€622, 172.3 tok/s on Llama-2 7B Q4_0, which works out to 246.1 tok/s per $1,000.
Card-only prices: you also need a PC around it. Ranked on Llama-2 7B Q4_0, the one test every card has. 24GB and 32GB cards don't fit gpt-oss-120b on their own.
| # | Machine | Price | tok/s | per $1k |
|---|---|---|---|---|
| 1 | NVIDIA RTX 3090 24GB (used) GPU · 24GB · checked 2026-10-16 | $700 ≈€622 | 172.3 | 246.1 |
tok/s = generation speed on Llama-2 7B Q4_0 from public benchmarks. per $1k = tok/s per $1,000 of the US price. ≈ = converted at the ECB rate.
Not ranked: NVIDIA RTX 5090 32GB (new) (no tracked price); NVIDIA RTX 4090 24GB (used) (no tracked price).
Per-model fit and speed for each card is on the GPUs page.
How we picked
Ranked by measured tokens per second per $1,000 of the current US price. Speeds are public benchmark results, one per chip, linked on each product page and on Benchmarks; engines and quants differ between sources. Prices checked 2026-10-16 19:02 UTC and refreshed every 6 hours.
FAQ
Which GPU is best value for local LLMs?
NVIDIA RTX 3090 24GB (used), on current card prices and public Llama-2 7B results. Only cards with a tracked price are ranked.
Is a 24GB GPU enough?
For models up to about 30B at 4-bit, yes. gpt-oss-120b and 70B models need more memory than one card has.
GPU or mini PC?
A card is faster on models that fit in its memory. A 128GB mini PC runs bigger models, slower. See the main ranking for complete machines.