RTX 3060 12 GB
12 GB on a 192-bit bus and the cheapest way into CUDA: enough for 7 to 14 B models at Q4, not enough for the ones above that.
Specs at NVIDIA ↗ Will it fit? ↗47 entries
Every card worth considering for local AI, with the numbers that decide everything: VRAM, memory bandwidth, power — and the hardware generation that decides which engines will run on it at all. Filter by vendor, sort by what you care about.
12 GB on a 192-bit bus and the cheapest way into CUDA: enough for 7 to 14 B models at Q4, not enough for the ones above that.
Specs at NVIDIA ↗ Will it fit? ↗Flagship silicon of its generation hobbled by 12 GB: fast memory, wrong capacity for local models.
Specs at NVIDIA ↗ Will it fit? ↗24 GB at 936 GB/s with an NVLink connector: the second-hand card most local builds are still built on.
Specs at NVIDIA ↗ Will it fit? ↗The fastest 24 GB card before the 40 series, and the least efficient of them: 450 W for 1,008 GB/s, no NVLink.
Specs at NVIDIA ↗ Will it fit? ↗16 GB in a 165 W card, but only 288 GB/s and a x8 PCIe link on most models: capacity without speed.
Specs at NVIDIA ↗ Will it fit? ↗16 GB at 672 GB/s: the middle of the consumer range, comfortable for 14 to 27 B models at Q4.
Specs at NVIDIA ↗ Will it fit? ↗736 GB/s with 16 GB of memory: quick in use, and the capacity decides what fits, not the speed.
Specs at NVIDIA ↗ Will it fit? ↗1,008 GB/s, the fastest consumer card of its generation: 24 GB is the ceiling and there is no NVLink.
Specs at NVIDIA ↗ Will it fit? ↗16 GB at 180 W on a x8 link: cheap capacity per watt with modest bandwidth, the trade a small box needs.
Specs at NVIDIA ↗ Will it fit? ↗Nearly 4090-class bandwidth on 16 GB: fast, and still capped by capacity rather than throughput.
Specs at NVIDIA ↗ Will it fit? ↗960 GB/s on 16 GB: a card whose speed is ahead of its memory, so large models still do not fit.
Specs at NVIDIA ↗ Will it fit? ↗1,792 GB/s and 32 GB: twice the 4090's bandwidth and the capacity that decides whether a model fits at all.
Specs at NVIDIA ↗ Will it fit? ↗24 GB of ECC on a passively cooled blower: the cheapest 24 GB per slot, held back by 347 GB/s and Pascal-era compute.
Will it fit? ↗48 GB of ECC at 768 GB/s in two slots: the quiet, warrantied route to 48 GB without modifying anything.
Specs at NVIDIA ↗ Will it fit? ↗48 GB at 960 GB/s: the stock alternative to a memory-modified consumer card, with a warranty.
Specs at NVIDIA ↗ Will it fit? ↗96 GB at 1,792 GB/s: a desktop card that holds a 70 B at Q8 or a 120 B mixture of experts with context.
Specs at NVIDIA ↗ Will it fit? ↗20 GB at 800 GB/s under ROCm: healthy bandwidth, and 20 GB sits between two model sizes rather than on one.
Specs at AMD ↗ Will it fit? ↗24 GB at 960 GB/s, the fastest RDNA 3 card: what runs is decided by ROCm support, not by the silicon.
Specs at AMD ↗ Will it fit? ↗RDNA 4 with 16 GB at 640 GB/s: more compute than memory bandwidth, so it serves small models quickly.
Specs at AMD ↗ Will it fit? ↗48 GB at 864 GB/s in a blower: the ROCm answer to a 48 GB workstation card.
Will it fit? ↗32 GB of HBM2 at 1,024 GB/s for very little second-hand, and dropped from current ROCm releases: cheap, and on its own.
Will it fit? ↗16 GB at 560 GB/s through SYCL or Vulkan: workable in llama.cpp, closed to CUDA-only tooling.
Will it fit? ↗12 GB at 456 GB/s on a tight budget: the entry point to Intel's stack.
Will it fit? ↗24 GB at 200 W: capacity per watt is the point of this card, bandwidth is not.
Will it fit? ↗128 GB unified with 96 GB the GPU can address, at 256 GB/s: enormous capacity, laptop-class bandwidth.
Specs at AMD ↗ Will it fit? ↗128 GB unified at 273 GB/s in a small box: capacity and CUDA, with bandwidth closer to a laptop than a GPU.
Specs at NVIDIA ↗ Will it fit? ↗The same GB10 box with half the memory: 64 GB unified at 273 GB/s, which NVIDIA quotes at up to 100 billion parameters, and two units cluster over QSFP to pool 128 GB. OEM-only from 23 October 2026, and the OEM systems list above what the 128 GB Founders Edition sells for.
Announcement at NVIDIA ↗ Will it fit? ↗546 GB/s of unified memory in a laptop: quiet, efficient, and capped at 128 GB.
Specs at Apple ↗ Will it fit? ↗819 GB/s with up to 512 GB unified: the largest single-machine working set that is not a rack.
Specs at Apple ↗ Will it fit? ↗Single-slot 16 GB ECC card drawing 140 W from one 6-pin connector, with a narrow 256-bit, 448 GB/s bus, so it is a capacity-and-power pick rather than a fast one.
Specs at NVIDIA ↗ Will it fit? ↗16 GB of GDDR6 at 320 GB/s in a 70 W half-height slot; designed for inference density, with the catch that 320 GB/s limits large-model throughput.
Specs at NVIDIA ↗ Will it fit? ↗An Ampere GA102 card rebuilt with 2 GB GDDR6X modules to reach 20 GB; bandwidth stays at the stock 760 GB/s on its 320-bit bus, and it is not a retail product.
Base card at NVIDIA ↗ Will it fit? ↗Ada-generation 20 GB ECC card at 130 W, limited to a 160-bit, 360 GB/s bus; useful for models that fit in 20 GB, not for bandwidth-bound throughput.
Specs at NVIDIA ↗ Will it fit? ↗A Turing TU102 card rebuilt with 2 GB modules to reach 22 GB; bandwidth stays at the stock 616 GB/s on its 352-bit bus, and it is a workshop modification rather than a retail product.
Will it fit? ↗24 GB in a 72 W low-profile slot at 300 GB/s; built for dense multi-GPU inference where power and slot count matter more than bandwidth.
Specs at NVIDIA ↗ Will it fit? ↗24 GB of ECC GDDR6 on a 768 GB/s bus at 230 W; an Ampere workstation card that gives 24 GB without the 350 W draw of an RTX 3090, at lower compute.
Specs at NVIDIA ↗ Will it fit? ↗32 GB of HBM2 at 900 GB/s on a 250 W PCIe card; Volta silicon that still runs CUDA but has only first-generation tensor cores and no BF16 support.
Specs at NVIDIA ↗ Will it fit? ↗PCIe A100 with 40 GB of ECC HBM2 at 1,555 GB/s and a 250 W board power; the smaller A100 memory tier, now widely available used, with no display outputs.
Specs at NVIDIA ↗ Will it fit? ↗An Ampere GA102 card rebuilt with 2 GB GDDR6X modules to reach 48 GB; bandwidth stays at the stock 936 GB/s on its 384-bit bus, and it is a workshop mod, not retail.
Base card at NVIDIA ↗ Will it fit? ↗48 GB of ECC GDDR6 at 864 GB/s on a 350 W PCIe card; the no-SXM route to 48 GB, but PCIe-only interconnect with no NVLink.
Specs at NVIDIA ↗ Will it fit? ↗An Ada AD102 card rebuilt with 3 GB GDDR6X modules to reach 48 GB; bandwidth stays at the stock 1,008 GB/s on its 384-bit bus, and only a few workshops build it.
Base card at NVIDIA ↗ Will it fit? ↗PCIe A100 with 80 GB of ECC HBM2e at 1,935 GB/s and 300 W; the SXM4 version of the same 80 GB part runs 2,039 GB/s at 400 W.
Specs at NVIDIA ↗ Will it fit? ↗80 GB of ECC HBM3 at 3,350 GB/s with board power up to 700 W in SXM form; the fastest 80 GB part here, but practically SXM-server-only and power-hungry.
Specs at NVIDIA ↗ Will it fit? ↗32 GB of GDDR6 on a 256-bit, 512 GB/s bus at 250 W; an RDNA2 workstation card with ROCm support whose bandwidth is low for its 32 GB capacity.
Will it fit? ↗32 GB of GDDR6 on a 256-bit bus at 640 GB/s and 300 W: the RDNA4 workstation card for local inference, where 32 GB of VRAM is the point and 640 GB/s is the ceiling.
Will it fit? ↗64 GB of full-chip-ECC HBM2e at 1,638 GB/s on a 300 W PCIe add-in card; the PCIe 64 GB option in this set, with ROCm as the software stack.
Specs at AMD ↗ Will it fit? ↗192 GB of ECC HBM3 at 5,325 GB/s in a 750 W OAM module; the largest single-GPU memory for local models, but it is an OAM part that needs a supported server and ROCm.
Specs at AMD ↗ Will it fit? ↗Bandwidth is the ceiling on generation speed: a card with 1,000 GB/s and one with 300 GB/s run the same model at roughly three times different speed, whatever the shader count says. The Q4_K_M figure is weights only (about 4.85 bits per parameter, no context cache) — check it in Will it fit? including the context you actually want. Bandwidth per watt is a rough efficiency signal for a machine left on all day, not a benchmark. Generation is the rung the card sits on — a compute capability (cc 8.9) on NVIDIA, a gfx target on AMD, a family name on Intel and Apple — and it is what an engine's stated minimum refers to: a card below it is refused or degraded whatever the memory says, as The architecture ladder sets out per vendor.
Memory-modified cards (24 GB → 48 GB) are listed separately, with who does the work: the 48 GB RTX 4090.
Fitting is not running. The three numbers above decide how much model a card holds and how fast it generates; a fourth decides whether the engine you picked will load on it at all — the card’s architecture generation, and the minimum the engine was compiled for. That is the subject of The architecture ladder: NVIDIA’s compute capability in one table, and what AMD, Intel and Apple use instead. Will it run then takes the card you are looking at and compares it with the floors every engine in the directory publishes, quoting the line each one comes from.
Compute capability 8.9, AMD gfx targets, Intel generations, Apple GPU families: what each step of GPU hardware added, which engines refuse below a step, and how to read your own card's step in one command.
NVIDIA never sold one. Who takes a 24 GB card to 48 GB, what it costs, what it breaks, and how to check a modified card before the money leaves your account.