47 entries

GPUs

Every card worth considering for local AI, with the numbers that decide everything: VRAM, memory bandwidth, power — and the hardware generation that decides which engines will run on it at all. Filter by vendor, sort by what you care about.

NVIDIA CUDA

RTX 3060 12 GB

12 GB on a 192-bit bus and the cheapest way into CUDA: enough for 7 to 14 B models at Q4, not enough for the ones above that.

Memory
12 GB
Bandwidth
360 GB/s
Power
170 W
Generation
cc 8.6
Fits at Q4_K_M
≈ 19 B parameters
Bandwidth per watt
2.1 GB/s/W
Specs at NVIDIA ↗ Will it fit? ↗
NVIDIA CUDA

RTX 3080 Ti

Flagship silicon of its generation hobbled by 12 GB: fast memory, wrong capacity for local models.

Memory
12 GB
Bandwidth
912 GB/s
Power
350 W
Generation
cc 8.6
Fits at Q4_K_M
≈ 19 B parameters
Bandwidth per watt
2.6 GB/s/W
Specs at NVIDIA ↗ Will it fit? ↗
NVIDIA CUDA

RTX 3090

24 GB at 936 GB/s with an NVLink connector: the second-hand card most local builds are still built on.

Memory
24 GB
Bandwidth
936 GB/s
Power
350 W
Generation
cc 8.6
Fits at Q4_K_M
≈ 39 B parameters
Bandwidth per watt
2.7 GB/s/W
Specs at NVIDIA ↗ Will it fit? ↗
NVIDIA CUDA

RTX 3090 Ti

The fastest 24 GB card before the 40 series, and the least efficient of them: 450 W for 1,008 GB/s, no NVLink.

Memory
24 GB
Bandwidth
1008 GB/s
Power
450 W
Generation
cc 8.6
Fits at Q4_K_M
≈ 39 B parameters
Bandwidth per watt
2.2 GB/s/W
Specs at NVIDIA ↗ Will it fit? ↗
NVIDIA CUDA

RTX 4060 Ti 16 GB

16 GB in a 165 W card, but only 288 GB/s and a x8 PCIe link on most models: capacity without speed.

Memory
16 GB
Bandwidth
288 GB/s
Power
165 W
Generation
cc 8.9
Fits at Q4_K_M
≈ 26 B parameters
Bandwidth per watt
1.7 GB/s/W
Specs at NVIDIA ↗ Will it fit? ↗
NVIDIA CUDA

RTX 4070 Ti Super

16 GB at 672 GB/s: the middle of the consumer range, comfortable for 14 to 27 B models at Q4.

Memory
16 GB
Bandwidth
672 GB/s
Power
285 W
Generation
cc 8.9
Fits at Q4_K_M
≈ 26 B parameters
Bandwidth per watt
2.4 GB/s/W
Specs at NVIDIA ↗ Will it fit? ↗
NVIDIA CUDA

RTX 4080 Super

736 GB/s with 16 GB of memory: quick in use, and the capacity decides what fits, not the speed.

Memory
16 GB
Bandwidth
736 GB/s
Power
320 W
Generation
cc 8.9
Fits at Q4_K_M
≈ 26 B parameters
Bandwidth per watt
2.3 GB/s/W
Specs at NVIDIA ↗ Will it fit? ↗
NVIDIA CUDA

RTX 4090

1,008 GB/s, the fastest consumer card of its generation: 24 GB is the ceiling and there is no NVLink.

Memory
24 GB
Bandwidth
1008 GB/s
Power
450 W
Generation
cc 8.9
Fits at Q4_K_M
≈ 39 B parameters
Bandwidth per watt
2.2 GB/s/W
Specs at NVIDIA ↗ Will it fit? ↗
NVIDIA CUDA

RTX 5060 Ti 16 GB

16 GB at 180 W on a x8 link: cheap capacity per watt with modest bandwidth, the trade a small box needs.

Memory
16 GB
Bandwidth
448 GB/s
Power
180 W
Generation
cc 12.0
Fits at Q4_K_M
≈ 26 B parameters
Bandwidth per watt
2.5 GB/s/W
Specs at NVIDIA ↗ Will it fit? ↗
NVIDIA CUDA

RTX 5070 Ti

Nearly 4090-class bandwidth on 16 GB: fast, and still capped by capacity rather than throughput.

Memory
16 GB
Bandwidth
896 GB/s
Power
300 W
Generation
cc 12.0
Fits at Q4_K_M
≈ 26 B parameters
Bandwidth per watt
3.0 GB/s/W
Specs at NVIDIA ↗ Will it fit? ↗
NVIDIA CUDA

RTX 5080

960 GB/s on 16 GB: a card whose speed is ahead of its memory, so large models still do not fit.

Memory
16 GB
Bandwidth
960 GB/s
Power
360 W
Generation
cc 12.0
Fits at Q4_K_M
≈ 26 B parameters
Bandwidth per watt
2.7 GB/s/W
Specs at NVIDIA ↗ Will it fit? ↗
NVIDIA CUDA

RTX 5090

1,792 GB/s and 32 GB: twice the 4090's bandwidth and the capacity that decides whether a model fits at all.

Memory
32 GB
Bandwidth
1792 GB/s
Power
575 W
Generation
cc 12.0
Fits at Q4_K_M
≈ 52 B parameters
Bandwidth per watt
3.1 GB/s/W
Specs at NVIDIA ↗ Will it fit? ↗
NVIDIA CUDA

Tesla P40

24 GB of ECC on a passively cooled blower: the cheapest 24 GB per slot, held back by 347 GB/s and Pascal-era compute.

Memory
24 GB
Bandwidth
347 GB/s
Power
250 W
Generation
cc 6.1
Fits at Q4_K_M
≈ 39 B parameters
Bandwidth per watt
1.4 GB/s/W
Will it fit? ↗
NVIDIA CUDA

RTX A6000

48 GB of ECC at 768 GB/s in two slots: the quiet, warrantied route to 48 GB without modifying anything.

Memory
48 GB
Bandwidth
768 GB/s
Power
300 W
Generation
cc 8.6
Fits at Q4_K_M
≈ 79 B parameters
Bandwidth per watt
2.6 GB/s/W
Specs at NVIDIA ↗ Will it fit? ↗
NVIDIA CUDA

RTX 6000 Ada

48 GB at 960 GB/s: the stock alternative to a memory-modified consumer card, with a warranty.

Memory
48 GB
Bandwidth
960 GB/s
Power
300 W
Generation
cc 8.9
Fits at Q4_K_M
≈ 79 B parameters
Bandwidth per watt
3.2 GB/s/W
Specs at NVIDIA ↗ Will it fit? ↗
NVIDIA CUDA

RTX PRO 6000 Blackwell

96 GB at 1,792 GB/s: a desktop card that holds a 70 B at Q8 or a 120 B mixture of experts with context.

Memory
96 GB
Bandwidth
1792 GB/s
Power
600 W
Generation
cc 12.0
Fits at Q4_K_M
≈ 158 B parameters
Bandwidth per watt
3.0 GB/s/W
Specs at NVIDIA ↗ Will it fit? ↗
AMD ROCm

Radeon RX 7900 XT

20 GB at 800 GB/s under ROCm: healthy bandwidth, and 20 GB sits between two model sizes rather than on one.

Memory
20 GB
Bandwidth
800 GB/s
Power
315 W
Generation
gfx1100 (RDNA3)
Fits at Q4_K_M
≈ 32 B parameters
Bandwidth per watt
2.5 GB/s/W
Specs at AMD ↗ Will it fit? ↗
AMD ROCm

Radeon RX 7900 XTX

24 GB at 960 GB/s, the fastest RDNA 3 card: what runs is decided by ROCm support, not by the silicon.

Memory
24 GB
Bandwidth
960 GB/s
Power
355 W
Generation
gfx1100 (RDNA3)
Fits at Q4_K_M
≈ 39 B parameters
Bandwidth per watt
2.7 GB/s/W
Specs at AMD ↗ Will it fit? ↗
AMD ROCm

Radeon RX 9070 XT

RDNA 4 with 16 GB at 640 GB/s: more compute than memory bandwidth, so it serves small models quickly.

Memory
16 GB
Bandwidth
640 GB/s
Power
304 W
Generation
gfx1201 (RDNA4)
Fits at Q4_K_M
≈ 26 B parameters
Bandwidth per watt
2.1 GB/s/W
Specs at AMD ↗ Will it fit? ↗
AMD ROCm

Radeon Pro W7900

48 GB at 864 GB/s in a blower: the ROCm answer to a 48 GB workstation card.

Memory
48 GB
Bandwidth
864 GB/s
Power
295 W
Generation
gfx1100 (RDNA3)
Fits at Q4_K_M
≈ 79 B parameters
Bandwidth per watt
2.9 GB/s/W
Will it fit? ↗
AMD ROCm

Instinct MI50 32 GB

32 GB of HBM2 at 1,024 GB/s for very little second-hand, and dropped from current ROCm releases: cheap, and on its own.

Memory
32 GB
Bandwidth
1024 GB/s
Power
300 W
Generation
gfx906 (GCN5.1)
Fits at Q4_K_M
≈ 52 B parameters
Bandwidth per watt
3.4 GB/s/W
Will it fit? ↗
Intel SYCL

Arc A770 16 GB

16 GB at 560 GB/s through SYCL or Vulkan: workable in llama.cpp, closed to CUDA-only tooling.

Memory
16 GB
Bandwidth
560 GB/s
Power
225 W
Generation
Xe-HPG (dg2)
Fits at Q4_K_M
≈ 26 B parameters
Bandwidth per watt
2.5 GB/s/W
Will it fit? ↗
Intel SYCL

Arc B580

12 GB at 456 GB/s on a tight budget: the entry point to Intel's stack.

Memory
12 GB
Bandwidth
456 GB/s
Power
190 W
Generation
Xe2 (bmg)
Fits at Q4_K_M
≈ 19 B parameters
Bandwidth per watt
2.4 GB/s/W
Will it fit? ↗
Intel SYCL

Arc Pro B60

24 GB at 200 W: capacity per watt is the point of this card, bandwidth is not.

Memory
24 GB
Bandwidth
456 GB/s
Power
200 W
Generation
Xe2 (bmg)
Fits at Q4_K_M
≈ 39 B parameters
Bandwidth per watt
2.3 GB/s/W
Will it fit? ↗
AMD Vulkan

Ryzen AI Max+ 395, 128 GB

128 GB unified with 96 GB the GPU can address, at 256 GB/s: enormous capacity, laptop-class bandwidth.

Memory
128 GB (96 GB usable, unified)
Bandwidth
256 GB/s
Power
140 W
Generation
gfx1151 (RDNA3.5)
Fits at Q4_K_M
≈ 158 B parameters
Bandwidth per watt
1.8 GB/s/W
Specs at AMD ↗ Will it fit? ↗
NVIDIA CUDA

DGX Spark, 128 GB

128 GB unified at 273 GB/s in a small box: capacity and CUDA, with bandwidth closer to a laptop than a GPU.

Memory
128 GB (112 GB usable, unified)
Bandwidth
273 GB/s
Power
240 W
Generation
cc 12.1
Fits at Q4_K_M
≈ 184 B parameters
Bandwidth per watt
1.1 GB/s/W
Specs at NVIDIA ↗ Will it fit? ↗
NVIDIA CUDA

DGX Spark, 64 GB

The same GB10 box with half the memory: 64 GB unified at 273 GB/s, which NVIDIA quotes at up to 100 billion parameters, and two units cluster over QSFP to pool 128 GB. OEM-only from 23 October 2026, and the OEM systems list above what the 128 GB Founders Edition sells for.

Memory
64 GB
Bandwidth
273 GB/s
Power
240 W
Generation
cc 12.1
Fits at Q4_K_M
≈ 105 B parameters
Bandwidth per watt
1.1 GB/s/W
Announcement at NVIDIA ↗ Will it fit? ↗
Apple Metal

Mac M4 Max, 128 GB

546 GB/s of unified memory in a laptop: quiet, efficient, and capped at 128 GB.

Memory
128 GB (96 GB usable, unified)
Bandwidth
546 GB/s
Power
140 W
Generation
Apple family 9
Fits at Q4_K_M
≈ 158 B parameters
Bandwidth per watt
3.9 GB/s/W
Specs at Apple ↗ Will it fit? ↗
Apple Metal

Mac M3 Ultra, 512 GB

819 GB/s with up to 512 GB unified: the largest single-machine working set that is not a rack.

Memory
512 GB (384 GB usable, unified)
Bandwidth
819 GB/s
Power
270 W
Generation
Apple family 9
Fits at Q4_K_M
≈ 633 B parameters
Bandwidth per watt
3.0 GB/s/W
Specs at Apple ↗ Will it fit? ↗
NVIDIA CUDA

RTX A4000

Single-slot 16 GB ECC card drawing 140 W from one 6-pin connector, with a narrow 256-bit, 448 GB/s bus, so it is a capacity-and-power pick rather than a fast one.

Memory
16 GB
Bandwidth
448 GB/s
Power
140 W
Generation
cc 8.6
Fits at Q4_K_M
≈ 26 B parameters
Bandwidth per watt
3.2 GB/s/W
Specs at NVIDIA ↗ Will it fit? ↗
NVIDIA CUDA

Tesla T4

16 GB of GDDR6 at 320 GB/s in a 70 W half-height slot; designed for inference density, with the catch that 320 GB/s limits large-model throughput.

Memory
16 GB
Bandwidth
320 GB/s
Power
70 W
Generation
cc 7.5
Fits at Q4_K_M
≈ 26 B parameters
Bandwidth per watt
4.6 GB/s/W
Specs at NVIDIA ↗ Will it fit? ↗
NVIDIA CUDA

RTX 3080 20 GB (modified)

An Ampere GA102 card rebuilt with 2 GB GDDR6X modules to reach 20 GB; bandwidth stays at the stock 760 GB/s on its 320-bit bus, and it is not a retail product.

Memory
20 GB
Bandwidth
760 GB/s
Power
320 W
Generation
cc 8.6
Fits at Q4_K_M
≈ 32 B parameters
Bandwidth per watt
2.4 GB/s/W
Base card at NVIDIA ↗ Will it fit? ↗
NVIDIA CUDA

RTX 4000 Ada

Ada-generation 20 GB ECC card at 130 W, limited to a 160-bit, 360 GB/s bus; useful for models that fit in 20 GB, not for bandwidth-bound throughput.

Memory
20 GB
Bandwidth
360 GB/s
Power
130 W
Generation
cc 8.9
Fits at Q4_K_M
≈ 32 B parameters
Bandwidth per watt
2.8 GB/s/W
Specs at NVIDIA ↗ Will it fit? ↗
NVIDIA CUDA

RTX 2080 Ti 22 GB (modified)

A Turing TU102 card rebuilt with 2 GB modules to reach 22 GB; bandwidth stays at the stock 616 GB/s on its 352-bit bus, and it is a workshop modification rather than a retail product.

Memory
22 GB
Bandwidth
616 GB/s
Power
250 W
Generation
cc 7.5
Fits at Q4_K_M
≈ 36 B parameters
Bandwidth per watt
2.5 GB/s/W
Will it fit? ↗
NVIDIA CUDA

L4

24 GB in a 72 W low-profile slot at 300 GB/s; built for dense multi-GPU inference where power and slot count matter more than bandwidth.

Memory
24 GB
Bandwidth
300 GB/s
Power
72 W
Generation
cc 8.9
Fits at Q4_K_M
≈ 39 B parameters
Bandwidth per watt
4.2 GB/s/W
Specs at NVIDIA ↗ Will it fit? ↗
NVIDIA CUDA

RTX A5000

24 GB of ECC GDDR6 on a 768 GB/s bus at 230 W; an Ampere workstation card that gives 24 GB without the 350 W draw of an RTX 3090, at lower compute.

Memory
24 GB
Bandwidth
768 GB/s
Power
230 W
Generation
cc 8.6
Fits at Q4_K_M
≈ 39 B parameters
Bandwidth per watt
3.3 GB/s/W
Specs at NVIDIA ↗ Will it fit? ↗
NVIDIA CUDA

Tesla V100 32 GB

32 GB of HBM2 at 900 GB/s on a 250 W PCIe card; Volta silicon that still runs CUDA but has only first-generation tensor cores and no BF16 support.

Memory
32 GB
Bandwidth
900 GB/s
Power
250 W
Generation
cc 7.0
Fits at Q4_K_M
≈ 52 B parameters
Bandwidth per watt
3.6 GB/s/W
Specs at NVIDIA ↗ Will it fit? ↗
NVIDIA CUDA

A100 40 GB PCIe

PCIe A100 with 40 GB of ECC HBM2 at 1,555 GB/s and a 250 W board power; the smaller A100 memory tier, now widely available used, with no display outputs.

Memory
40 GB
Bandwidth
1555 GB/s
Power
250 W
Generation
cc 8.0
Fits at Q4_K_M
≈ 65 B parameters
Bandwidth per watt
6.2 GB/s/W
Specs at NVIDIA ↗ Will it fit? ↗
NVIDIA CUDA

RTX 3090 48 GB (modified)

An Ampere GA102 card rebuilt with 2 GB GDDR6X modules to reach 48 GB; bandwidth stays at the stock 936 GB/s on its 384-bit bus, and it is a workshop mod, not retail.

Memory
48 GB
Bandwidth
936 GB/s
Power
350 W
Generation
cc 8.6
Fits at Q4_K_M
≈ 79 B parameters
Bandwidth per watt
2.7 GB/s/W
Base card at NVIDIA ↗ Will it fit? ↗
NVIDIA CUDA

L40S

48 GB of ECC GDDR6 at 864 GB/s on a 350 W PCIe card; the no-SXM route to 48 GB, but PCIe-only interconnect with no NVLink.

Memory
48 GB
Bandwidth
864 GB/s
Power
350 W
Generation
cc 8.9
Fits at Q4_K_M
≈ 79 B parameters
Bandwidth per watt
2.5 GB/s/W
Specs at NVIDIA ↗ Will it fit? ↗
NVIDIA CUDA

RTX 4090 48 GB (modified)

An Ada AD102 card rebuilt with 3 GB GDDR6X modules to reach 48 GB; bandwidth stays at the stock 1,008 GB/s on its 384-bit bus, and only a few workshops build it.

Memory
48 GB
Bandwidth
1008 GB/s
Power
450 W
Generation
cc 8.9
Fits at Q4_K_M
≈ 79 B parameters
Bandwidth per watt
2.2 GB/s/W
Base card at NVIDIA ↗ Will it fit? ↗
NVIDIA CUDA

A100 80 GB PCIe

PCIe A100 with 80 GB of ECC HBM2e at 1,935 GB/s and 300 W; the SXM4 version of the same 80 GB part runs 2,039 GB/s at 400 W.

Memory
80 GB
Bandwidth
1935 GB/s
Power
300 W
Generation
cc 8.0
Fits at Q4_K_M
≈ 131 B parameters
Bandwidth per watt
6.5 GB/s/W
Specs at NVIDIA ↗ Will it fit? ↗
NVIDIA CUDA

H100 80 GB SXM

80 GB of ECC HBM3 at 3,350 GB/s with board power up to 700 W in SXM form; the fastest 80 GB part here, but practically SXM-server-only and power-hungry.

Memory
80 GB
Bandwidth
3350 GB/s
Power
700 W
Generation
cc 9.0
Fits at Q4_K_M
≈ 131 B parameters
Bandwidth per watt
4.8 GB/s/W
Specs at NVIDIA ↗ Will it fit? ↗
AMD ROCm

Radeon Pro W6800

32 GB of GDDR6 on a 256-bit, 512 GB/s bus at 250 W; an RDNA2 workstation card with ROCm support whose bandwidth is low for its 32 GB capacity.

Memory
32 GB
Bandwidth
512 GB/s
Power
250 W
Generation
gfx1030 (RDNA2)
Fits at Q4_K_M
≈ 52 B parameters
Bandwidth per watt
2.0 GB/s/W
Will it fit? ↗
AMD ROCm

Radeon AI PRO R9700

32 GB of GDDR6 on a 256-bit bus at 640 GB/s and 300 W: the RDNA4 workstation card for local inference, where 32 GB of VRAM is the point and 640 GB/s is the ceiling.

Memory
32 GB
Bandwidth
640 GB/s
Power
300 W
Generation
gfx1201 (RDNA4)
Fits at Q4_K_M
≈ 52 B parameters
Bandwidth per watt
2.1 GB/s/W
Will it fit? ↗
AMD ROCm

Instinct MI210

64 GB of full-chip-ECC HBM2e at 1,638 GB/s on a 300 W PCIe add-in card; the PCIe 64 GB option in this set, with ROCm as the software stack.

Memory
64 GB
Bandwidth
1638 GB/s
Power
300 W
Generation
gfx90a (CDNA2)
Fits at Q4_K_M
≈ 105 B parameters
Bandwidth per watt
5.5 GB/s/W
Specs at AMD ↗ Will it fit? ↗
AMD ROCm

Instinct MI300X

192 GB of ECC HBM3 at 5,325 GB/s in a 750 W OAM module; the largest single-GPU memory for local models, but it is an OAM part that needs a supported server and ROCm.

Memory
192 GB
Bandwidth
5325 GB/s
Power
750 W
Generation
gfx942 (CDNA3)
Fits at Q4_K_M
≈ 316 B parameters
Bandwidth per watt
7.1 GB/s/W
Specs at AMD ↗ Will it fit? ↗

Bandwidth is the ceiling on generation speed: a card with 1,000 GB/s and one with 300 GB/s run the same model at roughly three times different speed, whatever the shader count says. The Q4_K_M figure is weights only (about 4.85 bits per parameter, no context cache) — check it in Will it fit? including the context you actually want. Bandwidth per watt is a rough efficiency signal for a machine left on all day, not a benchmark. Generation is the rung the card sits on — a compute capability (cc 8.9) on NVIDIA, a gfx target on AMD, a family name on Intel and Apple — and it is what an engine's stated minimum refers to: a card below it is refused or degraded whatever the memory says, as The architecture ladder sets out per vendor.

Memory-modified cards (24 GB → 48 GB) are listed separately, with who does the work: the 48 GB RTX 4090.

Fitting is not running. The three numbers above decide how much model a card holds and how fast it generates; a fourth decides whether the engine you picked will load on it at all — the card’s architecture generation, and the minimum the engine was compiled for. That is the subject of The architecture ladder: NVIDIA’s compute capability in one table, and what AMD, Intel and Apple use instead. Will it run then takes the card you are looking at and compares it with the floors every engine in the directory publishes, quoting the line each one comes from.

The architecture ladder: why a card that fits can still be refused

2026-10-07

Compute capability 8.9, AMD gfx targets, Intel generations, Apple GPU families: what each step of GPU hardware added, which engines refuse below a step, and how to read your own card's step in one command.

The 48 GB RTX 4090

2026-10-02

NVIDIA never sold one. Who takes a 24 GB card to 48 GB, what it costs, what it breaks, and how to check a modified card before the money leaves your account.