Garage · Updated 2026-10-10

An RTX 3090 on a 32-core Threadripper

by Need for VRAM @needforvram Verified owner

VRAM24 GB
GPUs1
Wall powerto measure
Costto add
Cards
1 × NVIDIA RTX 3090 (24 GB)
CPU
AMD Ryzen Threadripper 2990WX (32 cores, 64 threads)
System RAM
125 GB
Engines
llama.cpp (CUDA), vLLM, ComfyUI

Lap times

Generation speed, tokens/s
Qwen3-30B-A3B Q4_K_M 207.6 t/s

One 24 GB Ampere card that has to answer two different questions: serve a model over the network for the rest of the house, and render images when nobody is asking it for tokens. The card is used for those jobs one at a time, never both at once — the whole 24 GB goes to whichever one is running.

Why it is interesting

24 GB is the cheapest way to hold a 30B mixture-of-experts model at a useful quant, and this machine is what the leaderboard rows for the 3090 come from. The card runs at its stock 350 W limit: the numbers below are what the power limit buys, not what a tuned card would do.

The CPU is a 2990WX, 32 cores across four dies with a memory path that punishes anything that leaves one die, and 126 GB of system RAM. That combination is what makes partial offload survivable here: a model too big for the card still loads, slower.

What comes next

Wall power under load is not measured yet, and the machine has no meter of its own. Context-length sweeps (the same model at 8K, 16K and 32K) and an offload comparison against the full card are the next rows this rig can honestly produce.

The number that decides

The card is a hard ceiling, not a soft one: at 24 GB, the model either fits with its context cache or it does not, and no amount of system RAM changes that for a full offload.

This page belongs to Need for VRAM. Only they can change it, through a pull request on the site repository. Something wrong on this page? Report it.