51 entries · Checked 2026-10-10
Models
Open-weight models worth knowing, sorted by how much VRAM they need at 4-bit. Pick your card size to see what fits.
How the VRAM figure is computed
One rule for every model on this site: weights only, at Q4_K_M, which averages 4.85 bits per weight in llama.cpp.
GB = parameters (billions) × 4.85 ÷ 8
Mixture-of-experts models count all their parameters, since every expert has to sit in memory. Rounded to 0.5 GB below 10 GB, to 1 GB above. The context cache and runtime buffers come on top: Will it fit adds them for your context length and cards. Image and video models also need their text encoder.
Questions
- How much VRAM does a 27B model need at 4-bit?
- Roughly 0.57 GB per billion parameters at Q4_K_M, so about 15 to 16 GB of weights for 27B, plus the context cache: a few more GB at 32K with a q8_0 cache. Set your own card, quant and context in /will-it-fit/.
- Why is this list sorted by VRAM?
- Because memory is the constraint that decides whether a model runs at all on consumer hardware: size is the first filter, capability the second.
- Where do the parameter counts come from?
- From the safetensors metadata in the model’s own Hugging Face repository, not from the prose on the model card.