28 entries · Updated 2026-10-02
GPUs Every card worth considering for local AI, with the numbers that decide everything: VRAM, memory bandwidth and power. Filter by vendor, sort by what you care about.
All 28
17 6 3 2
Sort
Memory, most first Memory bandwidth, fastest first Bandwidth per watt, best first Power, lowest first Largest model that fits, biggest first Name, A to Z
CUDA RTX 3060 12 GB Memory 12 GB Bandwidth 360 GB/s Power 170 W Fits at Q4_K_M ≈ 19 B parameters Bandwidth per watt 2.1 GB/s/W CUDA RTX 3080 Ti Memory 12 GB Bandwidth 912 GB/s Power 350 W Fits at Q4_K_M ≈ 19 B parameters Bandwidth per watt 2.6 GB/s/W CUDA RTX 3090 Memory 24 GB Bandwidth 936 GB/s Power 350 W Fits at Q4_K_M ≈ 39 B parameters Bandwidth per watt 2.7 GB/s/W CUDA RTX 3090 Ti Memory 24 GB Bandwidth 1008 GB/s Power 450 W Fits at Q4_K_M ≈ 39 B parameters Bandwidth per watt 2.2 GB/s/W CUDA RTX 4060 Ti 16 GB Memory 16 GB Bandwidth 288 GB/s Power 165 W Fits at Q4_K_M ≈ 26 B parameters Bandwidth per watt 1.7 GB/s/W CUDA RTX 4070 Ti Super Memory 16 GB Bandwidth 672 GB/s Power 285 W Fits at Q4_K_M ≈ 26 B parameters Bandwidth per watt 2.4 GB/s/W CUDA RTX 4080 Super Memory 16 GB Bandwidth 736 GB/s Power 320 W Fits at Q4_K_M ≈ 26 B parameters Bandwidth per watt 2.3 GB/s/W CUDA RTX 4090 Memory 24 GB Bandwidth 1008 GB/s Power 450 W Fits at Q4_K_M ≈ 39 B parameters Bandwidth per watt 2.2 GB/s/W CUDA RTX 5060 Ti 16 GB Memory 16 GB Bandwidth 448 GB/s Power 180 W Fits at Q4_K_M ≈ 26 B parameters Bandwidth per watt 2.5 GB/s/W CUDA RTX 5070 Ti Memory 16 GB Bandwidth 896 GB/s Power 300 W Fits at Q4_K_M ≈ 26 B parameters Bandwidth per watt 3.0 GB/s/W CUDA RTX 5080 Memory 16 GB Bandwidth 960 GB/s Power 360 W Fits at Q4_K_M ≈ 26 B parameters Bandwidth per watt 2.7 GB/s/W CUDA RTX 5090 Memory 32 GB Bandwidth 1792 GB/s Power 575 W Fits at Q4_K_M ≈ 52 B parameters Bandwidth per watt 3.1 GB/s/W CUDA Tesla P40 Memory 24 GB Bandwidth 347 GB/s Power 250 W Fits at Q4_K_M ≈ 39 B parameters Bandwidth per watt 1.4 GB/s/W CUDA RTX A6000 Memory 48 GB Bandwidth 768 GB/s Power 300 W Fits at Q4_K_M ≈ 79 B parameters Bandwidth per watt 2.6 GB/s/W CUDA RTX 6000 Ada Memory 48 GB Bandwidth 960 GB/s Power 300 W Fits at Q4_K_M ≈ 79 B parameters Bandwidth per watt 3.2 GB/s/W CUDA RTX PRO 6000 Blackwell Memory 96 GB Bandwidth 1792 GB/s Power 600 W Fits at Q4_K_M ≈ 158 B parameters Bandwidth per watt 3.0 GB/s/W ROCm Radeon RX 7900 XT Memory 20 GB Bandwidth 800 GB/s Power 315 W Fits at Q4_K_M ≈ 32 B parameters Bandwidth per watt 2.5 GB/s/W ROCm Radeon RX 7900 XTX Memory 24 GB Bandwidth 960 GB/s Power 355 W Fits at Q4_K_M ≈ 39 B parameters Bandwidth per watt 2.7 GB/s/W ROCm Radeon RX 9070 XT Memory 16 GB Bandwidth 640 GB/s Power 304 W Fits at Q4_K_M ≈ 26 B parameters Bandwidth per watt 2.1 GB/s/W ROCm Radeon Pro W7900 Memory 48 GB Bandwidth 864 GB/s Power 295 W Fits at Q4_K_M ≈ 79 B parameters Bandwidth per watt 2.9 GB/s/W ROCm Instinct MI50 32 GB Memory 32 GB Bandwidth 1024 GB/s Power 300 W Fits at Q4_K_M ≈ 52 B parameters Bandwidth per watt 3.4 GB/s/W SYCL Arc A770 16 GB Memory 16 GB Bandwidth 560 GB/s Power 225 W Fits at Q4_K_M ≈ 26 B parameters Bandwidth per watt 2.5 GB/s/W SYCL Arc B580 Memory 12 GB Bandwidth 456 GB/s Power 190 W Fits at Q4_K_M ≈ 19 B parameters Bandwidth per watt 2.4 GB/s/W SYCL Arc Pro B60 Memory 24 GB Bandwidth 456 GB/s Power 200 W Fits at Q4_K_M ≈ 39 B parameters Bandwidth per watt 2.3 GB/s/W Vulkan Ryzen AI Max+ 395, 128 GB Memory 128 GB (96 GB usable, unified) Bandwidth 256 GB/s Power 140 W Fits at Q4_K_M ≈ 158 B parameters Bandwidth per watt 1.8 GB/s/W CUDA DGX Spark, 128 GB Memory 128 GB (112 GB usable, unified) Bandwidth 273 GB/s Power 240 W Fits at Q4_K_M ≈ 184 B parameters Bandwidth per watt 1.1 GB/s/W Metal Mac M4 Max, 128 GB Memory 128 GB (96 GB usable, unified) Bandwidth 546 GB/s Power 140 W Fits at Q4_K_M ≈ 158 B parameters Bandwidth per watt 3.9 GB/s/W Metal Mac M3 Ultra, 512 GB Memory 512 GB (384 GB usable, unified) Bandwidth 819 GB/s Power 270 W Fits at Q4_K_M ≈ 633 B parameters Bandwidth per watt 3.0 GB/s/W Bandwidth is the ceiling on generation speed: a card with 1,000 GB/s and one with 300 GB/s run the same model at roughly three times different speed, whatever the shader count says. The Q4_K_M figure is weights only (about 4.85 bits per parameter, no context cache) — check it in Will it fit? including the context you actually want. Bandwidth per watt is a rough efficiency signal for a machine left on all day, not a benchmark.
Memory-modified cards (24 GB → 48 GB) are listed separately, with who does the work: the 48 GB RTX 4090 .