Garage · Updated 2026-10-10
An RTX 3090 on a 32-core Threadripper
by Need for VRAM @needforvram Verified owner
- Cards
- 1 × NVIDIA RTX 3090 (24 GB)
- CPU
- AMD Ryzen Threadripper 2990WX (32 cores, 64 threads)
- System RAM
- 125 GB
- Engines
- llama.cpp (CUDA), vLLM, ComfyUI
Lap times
Generation speed, tokens/sOne 24 GB Ampere card that has to answer two different questions: serve a model over the network for the rest of the house, and render images when nobody is asking it for tokens. The card is used for those jobs one at a time, never both at once — the whole 24 GB goes to whichever one is running.
Why it is interesting
24 GB is the cheapest way to hold a 30B mixture-of-experts model at a useful quant, and this machine is what the leaderboard rows for the 3090 come from. The card runs at its stock 350 W limit: the numbers below are what the power limit buys, not what a tuned card would do.
The CPU is a 2990WX, 32 cores across four dies with a memory path that punishes anything that leaves one die, and 126 GB of system RAM. That combination is what makes partial offload survivable here: a model too big for the card still loads, slower.
What comes next
Wall power under load is not measured yet, and the machine has no meter of its own. Context-length sweeps (the same model at 8K, 16K and 32K) and an offload comparison against the full card are the next rows this rig can honestly produce.
The number that decides
The card is a hard ceiling, not a soft one: at 24 GB, the model either fits with its context cache or it does not, and no amount of system RAM changes that for a full offload.
This page belongs to Need for VRAM. Only they can change it, through a pull request on the site repository. Something wrong on this page? Report it.