· Checked 2026-10-10
Will it run?
Every floor this page uses, with its source
Read out of the engine's own code or documentation on 2026-10-07. No blog post, no benchmark table, no vendor slide.
| Engine | Backend | Line | Behaviour | Quote, from the project's own files | Source |
|---|---|---|---|---|---|
| ExLlamaV2 | CUDA, ROCm | no minimum published | info | To install from the repo you'll need the CUDA Toolkit and either gcc on Linux or (Build Tools for) Visual StudioNo minimum card is published anywhere in the repository: the README asks for a CUDA toolkit matching your platform, and the wheels are built per platform, Python version and CUDA version. Treat the floor as whatever that wheel was compiled for. | README.md |
| ExLlamaV3 | CUDA, ROCm | no minimum published | info | `exllamav3` declares a minimum `torch` version (>= 2.6.0) and CUDA version (>= 12.4), but beyond that the user is free to select a version of `torch` that is compatible with their environment.Version numbers are published, but they are toolkit numbers, not card numbers: a minimum torch and a minimum CUDA version. No card generation appears in the installation notes. | README.md |
| KoboldCpp | CUDA | none of its own: llama.cpp's backend | info | Inspired by **KoboldAI** and built on **llama.cpp**Its own README says what it is built on, so it has the ggml CUDA backend's floor rather than one of its own. | README.md |
| llama.cpp | CUDA | compute capability 5.0 | degrade | list(APPEND CMAKE_CUDA_ARCHITECTURES 50-virtual 61-virtual 70-virtual)The CUDA backend compiles kernels for 5.0 and up, so every card from Maxwell onwards has a kernel; below 5.0 there is none. The fast paths still need newer silicon: FP8 wants 8.9, and the newest attention kernels want 12.0. | ggml-cuda/CMakeLists.txt |
| llama.cpp | Metal | Metal GPU family 7 | degrade | dev->props.has_simdgroup_mm = [dev->mtl_device supportsFamily:MTLGPUFamilyApple7];The fast path is SIMD-group matrix multiply, which appears with the Metal family Apple7, the M1. An older Apple GPU or an Intel Mac still runs the Metal backend, without it. | ggml-metal-device.m |
| llama.cpp | ROCm | no minimum published | info | If your GPU is not officially supported you can use the environment variable [`HSA_OVERRIDE_GFX_VERSION`] set to a similar GPU, for example 10.3.0 on RDNA2 (e.g. gfx1030, gfx1031, or gfx1035)No minimum is published for the AMD path either: the documented route for a card the release does not build for is to make it answer as a supported sibling through an environment variable. | docs/build.md |
| llama.cpp | SYCL | no minimum published | info | The llama.cpp SYCL backend is primarily designed for **Intel GPUs**.The Intel path targets Intel GPUs through oneAPI and publishes no minimum generation of its own; the release builds name individual Arc parts instead. | docs/backend/SYCL.md |
| llama.cpp | Vulkan | no generation number: feature flags | info | if (strcmp("VK_KHR_cooperative_matrix", properties.extensionName) == 0) {The cross-vendor path has no generation number at all: it asks the device which optional extensions it exposes and enables each one separately, so the answer is a feature list rather than a rung on a ladder. | ggml-vulkan.cpp |
| MLX-LM | Metal | Apple silicon, macOS 14 or newer | refuse | - Using `Apple silicon <https://support.apple.com/en-us/116943>`_
- Using a native Python >= 3.10
- macOS >= 14.0The framework under MLX-LM is Apple-silicon only and needs macOS 14 or newer: an Intel Mac has no path, whatever its GPU. | docs/src/install.rst |
| Ollama | CUDA | none of its own: llama.cpp's backend | info | [llama.cpp](https://github.com/ggml-org/llama.cpp) project founded by Georgi Gerganov.Its README credits llama.cpp among the projects it is built on and publishes no minimum of its own; the practical floor is the ggml backend's. | README.md |
| SGLang | CUDA | families: Ampere, Ada, Hopper, Blackwell | refuse | | [NVIDIA](https://docs.sglang.io/docs/hardware-platforms/nvidia-gpus) | A100; H100/H200/H800/H20; B200/B300/GB200/GB300; select RTX 30/40/50 series, RTX 6000 Ada / PRO 6000; DGX Spark, Jetson Orin |The hardware table names families and some parts, and says «select RTX 30/40/50 series» rather than all of them: for a consumer card the honest answer is that the page does not name it. | README.md |
| SGLang | ROCm | families: CDNA3, CDNA4 | refuse | | [AMD](https://docs.sglang.io/docs/hardware-platforms/amd_gpu) | Instinct MI300X, MI325X, MI350X, MI355X |On AMD the project lists four Instinct parts, nothing else: Radeon cards are not in the table. | README.md |
| SGLang | SYCL | families: Xe, Xe2 | refuse | | Intel ([GPU](https://docs.sglang.io/docs/hardware-platforms/xpu) / [CPU](https://docs.sglang.io/docs/hardware-platforms/cpu_server)) | Arc / Arc Pro B-Series GPUs, Xeon CPUs |The Intel row names the Arc family, discrete cards only. | README.md |
| TensorFold | CUDA | compute capability 12.1 | refuse | const sms = b.option([]const u8, "sm", "SASS targets, comma separated (121; later 120,89)") orelse "121";One SASS image per target, and the default target is sm_121 — the GB10 in a DGX Spark, the only CUDA platform the README calls qualified. An Ada or Ampere card has no image to load until the sm_120 and sm_89 targets its build file lists as later arrive. The 8.9 minimum that applied before 1.0.0 belonged to the Python engine this rewrite replaced. | zig/build/cuda.zig |
| TensorFold | Metal | Apple silicon, M1 or newer | refuse | The macOS target is fixed to `aarch64-macos.13.0` with `apple_m1`, regardless of the builder's CPU or `-Dcpu`.The Mac build is pinned to apple_m1 and the README qualifies Metal on M1 through M5, so an Intel Mac has no path; Apple silicon does. | packaging/README.md |
| TensorRT-LLM | CUDA | families: Ampere, Ada, Hopper, Blackwell | refuse | TensorRT LLM supports the full spectrum of NVIDIA GPU architectures:
- **NVIDIA Blackwell**: B200, GB200, B300, GB300, DGX Spark
- **NVIDIA Hopper**: H100, H200, GH200
- **NVIDIA Ada Lovelace**: L20, L40/L40S
- **NVIDIA Ampere**: A100The supported-hardware page names four families and the parts inside them, datacenter parts: no GeForce card appears in it. A card whose family is named but whose part is not is marked as such rather than refused outright. | docs/source/supported-hardware.md |
| vLLM | CUDA · build list | compute capability 7.5 | refuse | set(CUDA_SUPPORTED_ARCHS "7.5;8.0;8.6;8.7;8.9;9.0;10.0;10.7;11.0;12.0")The build intersects the cards it finds with a supported list that depends on the toolkit version, and stops if nothing is left: «No supported CUDA architectures; the build would produce a binary with no usable kernels». The list starts at 7.5 with CUDA 12.8 and newer, at 7.0 with older toolkits. | CMakeLists.txt |
| vLLM | CUDA · bfloat16 at run time | compute capability 8.0 | warn | raise ValueError(
"Bfloat16 is only supported on GPUs "
"with compute capability of at least 8.0. " (bfloat16 weights)At run time this one raises instead: a bfloat16 model on a card under 8.0 stops with the message below. float16 still works through --dtype=half. | vllm/platforms/cuda.py |
| vLLM | ROCm | 17 gfx targets, gfx906 to gfx1201 | refuse | set(HIP_SUPPORTED_ARCHS "gfx906;gfx908;gfx90a;gfx942;gfx950;gfx1250;gfx1030;gfx1100;gfx1101;gfx1102;gfx1103;gfx1150;gfx1151;gfx1152;gfx1153;gfx1200;gfx1201")This is a membership test, not a ladder: the engine builds kernels for the targets in this list and for no others. A card outside it is refused even when a newer one is inside. | CMakeLists.txt |
| whisper.cpp | CUDA | compute capability 5.0 | degrade | list(APPEND CMAKE_CUDA_ARCHITECTURES 50-virtual 61-virtual 70-virtual)Same CUDA backend as llama.cpp, shipped in this repository: kernels from 5.0 up. | ggml-cuda/CMakeLists.txt |
A card that fits a model can still be refused by the engine. This page compares your card with the minimum hardware each engine publishes, quotes the line from the project’s own code or documentation, and says whether the engine stops, or merely runs without its fast paths.
The floors below are published, not inferred. Where a project says nothing, this page says nothing rather than guessing: an engine that publishes no minimum is listed as such, with the reason for the answer rather than a verdict.
Questions
- Why does the same card run one engine and not another?
- Because an engine is compiled for a set of hardware generations, and the projects do not agree on where the line sits. llama.cpp ships kernels from compute capability 5.0 up, so almost anything starts; TensorFold ships one SASS image per target and defaults to the newest one, so a card below it stops before a kernel loads; vLLM builds from a supported list and refuses to build when your card is not in it. The card did not change between those three answers, the compiled code did.
- Why does one engine appear twice?
- Because a project can publish more than one floor, and they do not fall in the same place. vLLM has a build-time list of CUDA architectures it will compile kernels for, and a separate run-time check that refuses bfloat16 weights below compute capability 8.0 — so the same card can be inside the build list and still lose that data type. The row says which of the two it is.
- My card has enough VRAM. Why is it still refused?
- Because fitting is a memory question and running is a compiled-code question. An engine built for a newer generation raises the moment it starts, before a single weight is read, however much memory you have. The ladder is on /gpus/architecture-ladder/ and the memory side is on /will-it-fit/.
- Why is AMD not one number here?
- Because AMD has two ladders that never meet: CDNA for Instinct and RDNA for Radeon. A gfx target is a version number, but a Radeon target and an Instinct target are not comparable, so this page never subtracts one from the other. On AMD the deciding test is membership: whether the engine builds kernels for your gfx target or not.
- An engine is missing from the list. Does that mean it runs?
- No, it means nothing can be checked. Three cases are left out on purpose: projects that publish no minimum at all, engines that hold no kernels of their own (routers, proxies, chat UIs — their floor is the floor of the engine behind them), and PyTorch-based stacks, where the floor comes from the torch wheel you installed rather than from the project.
- Does a 'runs' verdict mean it will be fast?
- No. A floor is the line where an engine starts, not where it performs. Speed comes from memory bandwidth, the quant you chose, the context length and the card’s fast paths, and those are on the card’s own page in the GPU directory.
- How do I check this myself?
- Every row carries the sentence it was read from and a link to the file. Run
nvidia-smi --query-gpu=compute_capon NVIDIA,rocminfo | grep gfxon AMD andsystem_profiler SPDisplaysDataTypeon a Mac, then compare with the number printed here.