Run AI models on your own hardware.
Find what to run, what to run on it, what it costs, and who to follow.
Checked 2026-10-10 · every page is read against the data behind it, and the date is printed with it
What fits in 24 GB
Full calculator →Weights at Q4_K_M: the largest models that fit in 24 GB are below. Add room for the context: the VRAM calculator does the full sum for your cards. Fitting is not running — the engine floors check the card against the minimum each engine publishes.
Kimi K3
1697 GB of weights at Q4_K_MMoonshot AI · 2.8T MoE, 104B active · Kimi K3 Licence
chatDeepSeek-V4-Pro
970 GB of weights at Q4_K_MDeepSeek · 1.6T MoE, 49B active · MIT
codeKimi K2.6
631 GB of weights at Q4_K_MMoonshot AI · 1.04T MoE, 32B active · Modified MIT
chatMistral Large 4
606 GB of weights at Q4_K_MMistral AI · 1T MoE, 49B active · Not stated in the announcement
codeGLM-5.3
457 GB of weights at Q4_K_MZ.ai · 753B MoE · GLM-5.3 Licence
codeGLM-5.1
451 GB of weights at Q4_K_MZ.ai · 744B MoE, 40B active · MIT
chatDeepSeek-V3.2
415 GB of weights at Q4_K_MDeepSeek · 685.36B MoE, 37B active · MIT
chatDeepSeek-V4.1-Flash
335 GB of weights at Q4_K_MDeepSeek · 552B MoE, 16B active · MIT
chatQwen3.8-Flash-Next
109 GB of weights at Q4_K_MQwen · 180B MoE, 6B active · Qwen Community
chatMistral Small 4
72 GB of weights at Q4_K_MMistral AI · 119B MoE, 6B active · Apache 2.0
codeLeanstral 1.5
72 GB of weights at Q4_K_MMistral AI · 119B MoE, 6B active · Apache 2.0
chatgpt-oss-120b
71 GB of weights at Q4_K_MOpenAI · 117B MoE, 5.1B active · Apache 2.0
chatLlama 3.3 70B
43 GB of weights at Q4_K_MMeta · 70.6B dense · Llama Community
chatQwen3.6-35B-A3B
21 GB of weights at Q4_K_MAlibaba · 35B MoE, 3B active · Apache 2.0
videoMiniMax-H3
20 GB of weights at Q4_K_MMiniMax · 33B dense · MiniMax H3 Community Licence
chatQwen3 32B
20 GB of weights at Q4_K_MQwen · 32.76B dense · Apache 2.0
chatGemma 4 31B
19 GB of weights at Q4_K_MGoogle · 31B dense · Apache 2.0
chatQwen3 30B A3B Instruct 2507
19 GB of weights at Q4_K_MQwen · 30.53B MoE, 3.3B active · Apache 2.0
codeQwen3 Coder 30B A3B
19 GB of weights at Q4_K_MQwen · 30.53B MoE, 3.3B active · Apache 2.0
chatQwen3.8-27B
17 GB of weights at Q4_K_MQwen · 27.8B dense · Apache 2.0
chatClef
17 GB of weights at Q4_K_MCloudflare · 27.36B dense · Apache 2.0
codeQwen3.6-27B
16 GB of weights at Q4_K_MAlibaba · 27B dense · Apache 2.0
videoWan 2.2 T2V-A14B
16 GB of weights at Q4_K_MAlibaba · 27B MoE, 14B active · Apache 2.0
chatTernary-Bonsai-2-27B
16 GB of weights at Q4_K_Mprism-ml · 27B dense · Apache 2.0 (the GGUF repository)
chatGemma 4 26B-A4B
16 GB of weights at Q4_K_MGoogle · 26B MoE, 4B active · Apache 2.0
videoLTX-2.5
13 GB of weights at Q4_K_MLightricks · 22B dense · Custom licence on the model card
chatgpt-oss-20b
13 GB of weights at Q4_K_MOpenAI · 21B MoE, 3.6B active · Apache 2.0
imageQwen-Image
12 GB of weights at Q4_K_MQwen · 20B dense · Apache 2.0
imageQwen-Image-2.1
10 GB of weights at Q4_K_MQwen · 16.5B dense · Qwen Research
chatMinistral 3 14B
8.5 GB of weights at Q4_K_MMistral AI · 14B dense · Apache 2.0
videoLTX-Video 13B
8 GB of weights at Q4_K_MLightricks · 13B dense · LTX Open Weights
videoHunyuanVideo
8 GB of weights at Q4_K_MTencent · 13B dense · Tencent Community Licence
imageFLUX.1 [dev]
7.5 GB of weights at Q4_K_MBlack Forest Labs · 12B dense · Non-commercial
chatGemma 4 12B
7.5 GB of weights at Q4_K_MGoogle · 12B dense · Apache 2.0
imageFLUX.1 schnell
7.5 GB of weights at Q4_K_MBlack Forest Labs · 12B dense · Apache 2.0
chatZDTaichu5.0-9B
6 GB of weights at Q4_K_MTaichuAI · 9.79B dense · Not stated on the card
chatClef Flash
5.5 GB of weights at Q4_K_MCloudflare · 9.41B dense · Apache 2.0
chatQwen3 8B
5 GB of weights at Q4_K_MQwen · 8.19B dense · Apache 2.0
imageStable Diffusion 3.5 Large
5 GB of weights at Q4_K_MStability AI · 8.1B dense · Stability Community Licence
chatLlama 3.1 8B
5 GB of weights at Q4_K_MMeta · 8.03B dense · Llama Community
chatMinistral 8B
5 GB of weights at Q4_K_MMistral AI · 8.02B dense · Non-commercial (Mistral Research)
chatGemma 4 E4B
5 GB of weights at Q4_K_MGoogle · 8B dense · Apache 2.0
videoWan 2.2 TI2V-5B
3 GB of weights at Q4_K_MAlibaba · 5B dense · Apache 2.0
chatQwen3 4B Instruct 2507
2.5 GB of weights at Q4_K_MQwen · 4.02B dense · Apache 2.0
chatPhi-4-mini
2.5 GB of weights at Q4_K_MMicrosoft · 3.84B dense · MIT
chatShieldstral 1.0
2 GB of weights at Q4_K_MMistral AI · 3B dense · Apache 2.0
imageStable Diffusion XL base 1.0
1.5 GB of weights at Q4_K_MStability AI · 2.6B dense · OpenRAIL++
speechWhisper large-v3
1 GB of weights at Q4_K_MOpenAI · 1.55B dense · MIT
speechParakeet TDT 0.6B v3
0.5 GB of weights at Q4_K_MNVIDIA · 0.6B dense · CC-BY-4.0
speechNemotron-3-Diarization
0 GB of weights at Q4_K_MNVIDIA · 0.099B dense · OpenMDW-1.1
speechKokoro 82M
0 GB of weights at Q4_K_Mhexgrad · 0.082B dense · Apache 2.0
Next on the calendar
All events →Open Source AI Week
Kubernetes Community Days UK, Edinburgh
PyTorch Conference North America
People and channels
All people and channels →Georgi Gerganov
@ggerganovCreated llama.cpp and the GGML/GGUF stack that most local inference still runs on.
X buildersComfyUI (comfyanonymous)
comfyanonymousCreated ComfyUI, the node editor most people use to run image and video models locally.
GitHub hardwareAnush Elangovan
@AnushElangovanRuns the ROCm Radeon team at AMD. The person to follow for where ROCm on Radeon is heading, and to ask for it.
X hardwareDigital Spaceport
@DigitalSpaceportBuilds quad and eight-GPU home servers on camera, with parts lists, power draw and long-term reviews.
YouTube quantizersDaniel Han
@danielhanchenCo-founder of Unsloth. Single-GPU fine-tuning, dynamic quants, and frequent fixes for day-one release bugs.
X researchersTim Dettmers
@Tim_Dettmersbitsandbytes and QLoRA. A lot of what we know about 4-bit on consumer GPUs started with him.
XFrom the garage
All rigs →A Radeon AI PRO R9700 next to a 3060
AMD Radeon AI PRO R9700 + RTX 3060 44 GB 2 GPUs Updated 2026-10-10 @needforvram ✓An RTX 3090 on a 32-core Threadripper
RTX 3090 24 GB 1 GPU Updated 2026-10-10 @needforvram ✓Threadripper with a mismatched pair
RTX 3080 Ti + RTX 3060 24 GB 2 GPUs Updated 2026-10-01Rig measurements
The leaderboard →| Rig | GPUs | Model | Engine | pp512 t/s | tg128 t/s | Wall W | Date |
|---|---|---|---|---|---|---|---|
| An RTX 3090 on a 32-core Threadripper | NVIDIA RTX 3090 24 GB | Qwen3-30B-A3B Qwen3-30B-A3B-Q4_K_M.gguf | llama.cpp b10002 (a7312ae) | 4232 | 207.6 | – | 2026-10-10 |
| A Radeon AI PRO R9700 next to a 3060 | AMD Radeon AI PRO R9700 32 GB | Qwen3-4B-Instruct-2507 Qwen3-4B-Instruct-2507-Q4_K_M.gguf | llama.cpp b11235 (6c7a87f) | 6478 | 139.6 | – | 2026-10-10 |
Follow the paddock
New models, rig tours and benchmarks. Posted on X, filmed for YouTube.
Why this site exists
Almost everyone who wants to keep up with AI hits a wall, and it is rarely only the graphics card. The model is a few gigabytes too big. The quant that fits answers worse. The runtime wants a driver you do not have. The tokens per second are fine until the context grows. The power draw, the price of the card, the weeks it takes to learn any of it - all of that decides whether you are in the game or watching it from outside.
The people who get through do it with the same handful of things: a model worth downloading, a box that can hold it, a runtime that works that week, someone who hit the same bug first, a cost they can justify, and an event where the people who built the tools are standing in the same room.
This site keeps track of those things - the models, the hardware and what fits where, the runtimes and their quirks, storage, the tools and what running them actually costs, the people and companies behind them, the books worth reading, the rigs people really built, and where everyone meets next. The list stays short on purpose: an entry is here because it helps someone who does not intend to miss this.
Missing someone? Suggest an entry.