51 entries · Checked 2026-10-10

Models

Open-weight models worth knowing, sorted by how much VRAM they need at 4-bit. Pick your card size to see what fits.

Your VRAM
speech
0.082B dense · Apache 2.0
text to speech, StyleTTS2 decoder, runs on CPU or GPU through ONNX
under 1 GB
·
speech
0.099B dense · OpenMDW-1.1
speaker diarization: who spoke when, not what was said. 99M parameters, about 100 MB in F32, runs through audio.cpp or NeMo rather than llama.cpp
under 1 GB
·
speech
0.6B dense · CC-BY-4.0
about 2 GB to load, multilingual ASR, runs through NeMo or ggml rather than llama.cpp
under 1 GB
·
speech
1.55B dense · MIT
1 GB
·
image
2.6B dense · OpenRAIL++
2.6B UNet is the largest part, plus two CLIP encoders (~0.8B) and a VAE
1.5 GB
·
chat
3B dense · Apache 2.0
policy-adaptive multimodal safety classifier for text and image moderation; the announcement says it runs on a single 16 GB GPU
2 GB
·
Phi-4-mini
Microsoft · GGUF ↗
chat
3.84B dense · MIT
2.5 GB
·
chat
4.02B dense · Apache 2.0
2.5 GB
·
video
5B dense · Apache 2.0
plus text encoder and VAE
3 GB
·
chat
8B dense · Apache 2.0
4B effective, the rest are per-layer embeddings
5 GB
·
Ministral 8B
Mistral AI · GGUF ↗
chat
8.02B dense · Non-commercial (Mistral Research)
3:1 sliding-window layers, so a 32K cache stays small at the cost of licence terms
5 GB
·
chat
8.03B dense · Llama Community
128K context and the widest tooling support of any 8B model; the Hugging Face page is gated (accept the licence before downloading)
5 GB
·
image
8.1B dense · Stability Community Licence
fp16 checkpoint is about 16.5 GB, plus a T5-XXL encoder; commercial use under a revenue cap; the Hugging Face page is gated (accept the licence before downloading)
5 GB
·
chat
8.19B dense · Apache 2.0
5 GB
·
Clef Flash
Cloudflare · GGUF ↗
chat
9.41B dense · Apache 2.0
the smaller sibling of Clef, published the same day under the same Apache 2.0 terms; 9.4B in BF16
5.5 GB
·
chat
9.79B dense · Not stated on the card
takes text and images; 9.8B in BF16; the org publishes its own GGUF builds from Q4_K_M up to BF16
6 GB
·
FLUX.1 [dev]
Black Forest Labs · GGUF ↗
image
12B dense · Non-commercial
plus T5 text encoder; the Hugging Face page is gated (accept the licence before downloading)
7.5 GB
·
chat
12B dense · Apache 2.0
7.5 GB
·
FLUX.1 schnell
Black Forest Labs · GGUF ↗
image
12B dense · Apache 2.0
plus a T5-XXL text encoder (~4.7B) and CLIP; the Apache-2.0 member of the family; the Hugging Face page is gated (accept the licence before downloading)
7.5 GB
·
LTX-Video 13B
Lightricks · GGUF ↗
video
13B dense · LTX Open Weights
a 2B variant and fp8 builds exist for smaller cards
8 GB
·
video
13B dense · Tencent Community Licence
the card quotes 60 GB minimum and 80 GB recommended; 45 GB at 544x960
8 GB
·
chat
14B dense · Apache 2.0
8.5 GB
·
image
16.5B dense · Qwen Research
7.1B transformer plus an 8.8B vision-language text encoder and a VAE: 33 GB of files, so a run needs more than that
10 GB
·
image
20B dense · Apache 2.0
plus a 7B Qwen2.5-VL text encoder and a VAE, so a run needs more than 20 GB
12 GB
·
chat
21B MoE, 3.6B active · Apache 2.0
13 GB
·
LTX-2.5
Lightricks · GGUF ↗
video
22B dense · Custom licence on the model card
22B transformer in dev and distilled versions, plus a Gemma-4 12B text encoder and its own VAE; the repo also ships int8 and nvfp4 builds. The Hugging Face page is gated (accept the licence before downloading)
13 GB
·
chat
26B MoE, 4B active · Apache 2.0
16 GB
·
code
27B dense · Apache 2.0
hybrid attention, only 16 of 64 layers keep a KV cache
16 GB
·
video
27B MoE, 14B active · Apache 2.0
MoE diffusion: the card's 720p command assumes 80 GB, offload flags bring it down
16 GB
·
chat
27B dense · Apache 2.0 (the GGUF repository)
ternary weights (1.58-bit post-training quantisation): the GGUF repository carries F16 and PQ2_0 builds and is what most people run, while the base weights are not public
16 GB
·
Clef
Cloudflare · GGUF ↗
chat
27.36B dense · Apache 2.0
27B multimodal model (text and images) released under Apache 2.0; bartowski and ggml-org both publish GGUF builds
17 GB
·
chat
27.8B dense · Apache 2.0
hybrid attention, only 16 of 64 layers keep a full KV cache
17 GB
·
chat
30.53B MoE, 3.3B active · Apache 2.0
MoE: 30B must sit in memory, only 3B run per token
19 GB
·
code
30.53B MoE, 3.3B active · Apache 2.0
MoE: same 30B memory footprint, built for agentic coding
19 GB
·
chat
31B dense · Apache 2.0
19 GB
·
chat
32.76B dense · Apache 2.0
Dense, not a MoE: all 32B run for every token
20 GB
·
video
33B dense · MiniMax H3 Community Licence
generates video and audio together; a run also needs a Qwen3-VL-32B text encoder and two VAEs, and the FL2VA and Ref2VA variants ship separately
20 GB
·
chat
35B MoE, 3B active · Apache 2.0
21 GB
·
chat
70.6B dense · Llama Community
the Hugging Face page is gated: accept the licence before downloading
43 GB
·
chat
117B MoE, 5.1B active · Apache 2.0
71 GB
·
chat
119B MoE, 6B active · Apache 2.0
72 GB
·
Leanstral 1.5
Mistral AI · GGUF ↗
code
119B MoE, 6B active · Apache 2.0
formal-verification model for Lean 4, also served as a free API; no official GGUF yet, so the link is a community build
72 GB
·
chat
180B MoE, 6B active · Qwen Community
125B of transformer plus a 51B n-gram embedding table and a 4B MTP head, 6B active per token; 12 of 48 layers keep a full KV cache
109 GB
·
chat
552B MoE, 16B active · MIT
multimodal and 1M context; the sparse-attention KV compression keeps the real cache far below the figure above
335 GB
·
chat
685.36B MoE, 37B active · MIT
MLA attention: the cache keeps a compressed latent, so the KV figures above overstate the real cache
415 GB
·
code
744B MoE, 40B active · MIT
451 GB
·
code
753B MoE · GLM-5.3 Licence
MoE: 256 experts of which 8 run per token; shipped in FP8, so the repo is 756 GB
457 GB
·
Mistral Large 4
Mistral AI
chat
1T MoE, 49B active · Not stated in the announcement
hybrid instruct-and-reasoning MoE with multimodal input; the open weights are not downloadable yet, the announcement says they drop at the end of the month
606 GB
·
Kimi K2.6
Moonshot AI · GGUF ↗
code
1.04T MoE, 32B active · Modified MIT
631 GB
·
chat
1.6T MoE, 49B active · MIT
1M context, FP4 experts with FP8 elsewhere: the release repo is 865 GB and the smallest GGUF is a Q3_K_M at 749 GB
970 GB
·
Kimi K3
Moonshot AI · GGUF ↗
chat
2.8T MoE, 104B active · Kimi K3 Licence
MLA attention: the cache keeps a compressed latent, so the KV figures above overstate the real cache; takes image and video input
1697 GB
·

How the VRAM figure is computed

One rule for every model on this site: weights only, at Q4_K_M, which averages 4.85 bits per weight in llama.cpp.

GB = parameters (billions) × 4.85 ÷ 8

Mixture-of-experts models count all their parameters, since every expert has to sit in memory. Rounded to 0.5 GB below 10 GB, to 1 GB above. The context cache and runtime buffers come on top: Will it fit adds them for your context length and cards. Image and video models also need their text encoder.

Questions

How much VRAM does a 27B model need at 4-bit?
Roughly 0.57 GB per billion parameters at Q4_K_M, so about 15 to 16 GB of weights for 27B, plus the context cache: a few more GB at 32K with a q8_0 cache. Set your own card, quant and context in /will-it-fit/.
Why is this list sorted by VRAM?
Because memory is the constraint that decides whether a model runs at all on consumer hardware: size is the first filter, capability the second.
Where do the parameter counts come from?
From the safetensors metadata in the model’s own Hugging Face repository, not from the prose on the model card.