Can you run Kimi K3 locally?
moonshotai/Kimi-K3 · 2,780B parameters · license kimi-k3 · on Hugging Face since 2026-06-13
Short answer: not on one tracked machine
The weights alone need about 1,685 GB at Q4_K_M and 2,954 GB at Q8_0 (estimates from parameter count × bits per weight). The largest tracked machine has 256GB, so no single box here can hold it at 4-bit.
What it would take (estimate)
Assumes 4-bit weights × 1.15 + 2 GB ≈ 1,940 GB, spread over machines with 85% of their memory usable. Clustering needs software such as llama.cpp RPC and fast networking; we have no public speed result for this model on any tracked setup.
| Option | Hardware cost | EUR |
|---|---|---|
| 18 × Framework Desktop 128GB (DIY) (128GB each) | ≈$62,082 | ≈€70,002 |
| 9 × Mac Studio M5 Ultra 80-core GPU, 256GB / 1TB (256GB each) | ≈$97,191 | ≈€112,761 |
Or rent it
| Cloud GPUs (80GB each) | USD | EUR |
|---|---|---|
| 25 × H100 80GB, RunPod (Community Cloud (Secure $2.89)) | ≈$49.75/hour | ≈€44.21/hour |
| 25 × H100 80GB, Vast.ai (median of 2 live offers) | ≈$77.58/hour | ≈€68.94/hour |
Hourly cost = GPUs needed × tracked H100 PCIe hourly price (see Rent instead). Multi-GPU machines with that many cards may not be offered at this price.
Smaller siblings that fit
No smaller sibling in our model list yet.
Why it's listed: 11.7k likes, 1M downloads.