NVIDIA · workstation

RTX 4000 Ada Generation

RTX 4000 Ada Generation has 20 GB of VRAM at 360 GB/s — about 18.60 GiB usable after driver and compositor overhead. 1953 of 2118 indexed models fit at 4K context with q4_0 KV.

Spec sheet· bandwidth, theoreticalFrom the file· fit from summed bytesPredicted· speed
Memory
20 GB
GDDR6
Bandwidth
360 GB/s
160-bit bus
Tensor FP16
107 TF
dense
TDP
130 W
$1250 MSRP
Context4K8K16K32K64K128K
KV cachef16q8_0q4_0quantizing the KV cache is a ~2× lever on the dominant term at long context
text 1676vision language 173video 16image 2audio asr 39audio tts 21embedding 26

What fits at 4K context

largest quantization that fits, per model · 1953 of 2118 indexed
ModelBest quantParamsWeights●KV●Total in memory◐Headroom◐tok/s◐
EuroLLM-22B-Instruct-2512Q6_K22.6B17.30 GiB0.24 GiB18.60 GiB0.00 GiB12±22%
Huihui-gemma-4-26B-A4B-it-abliteratedMoEUD-Q5_K_S26.5B17.48 GiB0.13 GiB18.60 GiB0.00 GiB12±22%
Huihui-Qwen3.5-35B-A3B-abliteratedMoEIQ4_XS36.0B17.57 GiB0.02 GiB18.59 GiB0.01 GiB71±37%
Qwen3.5-35B-A3B-BaseMoEIQ4_XS36.0B17.57 GiB0.02 GiB18.59 GiB0.01 GiB71±37%
GLM-4.7-Flash-DerestrictedMoEI1-Q4_131.2B17.52 GiB0.06 GiB18.58 GiB0.02 GiB54±37%
Huihui-GLM-4.7-Flash-abliteratedMoEI1-Q4_131.2B17.52 GiB0.06 GiB18.58 GiB0.02 GiB54±37%
GLM-Z1-32B-0414-uncensored-heretic-v2Q4_K_S32.6B17.42 GiB0.07 GiB18.58 GiB0.02 GiB12±22%
Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16UD-IQ3_S33.0B17.53 GiB0.00 GiB18.57 GiB0.03 GiB12±22%
GLM-4-32B-0414-Korean-CultureI1-Q4_K_S32.6B17.41 GiB0.07 GiB18.57 GiB0.03 GiB12±22%
GLM-Z1-32B-0414Q4_K_S32.6B17.41 GiB0.07 GiB18.57 GiB0.03 GiB12±22%
GLM-4-32B-0414Q4_K_S32.6B17.41 GiB0.07 GiB18.57 GiB0.03 GiB12±22%
GRM-2.6-Plus-0628Q4_K_L27.8B17.43 GiB0.07 GiB18.56 GiB0.04 GiB12±22%
ThinkingCap-Qwen3.6-27BQ4_K_L27.4B17.43 GiB0.07 GiB18.56 GiB0.04 GiB12±22%
Tess-4-27BQ4_K_L27.8B17.43 GiB0.07 GiB18.56 GiB0.04 GiB12±22%
Skywork-R1V3-38BQ4_K_S38.4B17.49 GiB0.00 GiB18.56 GiB0.04 GiB12±22%
Yi-34B-200K-DARE-megamerge-v8I1-IQ4_XS34.4B17.21 GiB0.26 GiB18.55 GiB0.05 GiB12±22%
dolphin-2.9.1-yi-1.5-34bI1-IQ4_XS34.4B17.21 GiB0.26 GiB18.55 GiB0.05 GiB12±22%
OrionStar-Yi-34B-Chat-LlamaI1-IQ4_XS34.4B17.21 GiB0.26 GiB18.55 GiB0.05 GiB12±22%
Yi-34B-200K-LlamafiedI1-IQ4_XS34.4B17.21 GiB0.26 GiB18.55 GiB0.05 GiB12±22%
Nous-Hermes-2-Yi-34BI1-IQ4_XS34.4B17.21 GiB0.26 GiB18.55 GiB0.05 GiB12±22%
Merged-RP-Stew-V2-34BI1-IQ4_XS34.4B17.21 GiB0.26 GiB18.55 GiB0.05 GiB12±22%
Capybara-Tess-Yi-34B-200KI1-IQ4_XS34.4B17.21 GiB0.26 GiB18.55 GiB0.05 GiB12±22%
GLM-4.7-Flash-hereticMoEQ4_K_L29.9B17.49 GiB0.06 GiB18.55 GiB0.05 GiB54±37%
Qwen3-42B-A3B-2507-Thinking-Abliterated-uncensored-TOTAL-RECALL-v2-Medium-MASTER-CODERMoEI1-IQ3_M42.4B17.41 GiB0.15 GiB18.55 GiB0.05 GiB52±37%
Aurora-Code-1MoEIQ4_XS34.7B17.51 GiB0.02 GiB18.54 GiB0.06 GiB71±37%
grug-35bMoEIQ4_XS35.1B17.51 GiB0.02 GiB18.54 GiB0.06 GiB71±37%
WorldSim-Opus-3.6-35B-A3BMoEIQ4_XS35.1B17.51 GiB0.02 GiB18.54 GiB0.06 GiB71±37%
Qwen3.6-35B-A3B-AnkoMoEIQ4_XS35.1B17.51 GiB0.02 GiB18.54 GiB0.06 GiB71±37%
KAT-Coder-V2.5-DevMoEIQ4_XS34.7B17.51 GiB0.02 GiB18.54 GiB0.06 GiB71±37%
Ornith-1.0-35BMoEIQ4_XS34.7B17.51 GiB0.02 GiB18.54 GiB0.06 GiB71±37%
Nex-N2-miniMoEIQ4_XS35.1B17.51 GiB0.02 GiB18.54 GiB0.06 GiB71±37%
North-Mini-Code-1.0MoEQ4_K_M30.5B17.46 GiB0.11 GiB18.54 GiB0.06 GiB53±37%
Gemma4-Gutenberg-31BQ4_K_S31.3B16.95 GiB0.51 GiB18.53 GiB0.07 GiB12±22%
gemma-4-31B-itQ4_K_S31.3B16.95 GiB0.51 GiB18.53 GiB0.07 GiB12±22%
Gemma4-Gutenberg-31B-HereticQ4_K_S31.3B16.95 GiB0.51 GiB18.53 GiB0.07 GiB12±22%
Equinox-31BQ4_K_S31.3B16.95 GiB0.51 GiB18.53 GiB0.07 GiB12±22%
gemma-4-31B-it-SDFT-Heretic-RPQ4_K_S30.7B16.95 GiB0.51 GiB18.53 GiB0.07 GiB12±22%
Qwen3.6-27B-uncensored-heretic-v2Q5_K_S27.4B17.40 GiB0.07 GiB18.53 GiB0.07 GiB12±22%
Qwen3.5-27B-Engineer-Deckard-GeminiI1-Q5_K_S27.7B17.40 GiB0.07 GiB18.53 GiB0.07 GiB12±22%
Qwen3.5-27B-HERETIC-Polaris-Advanced-Thinking-Alpha-uncensoredI1-Q5_K_S27.4B17.40 GiB0.07 GiB18.53 GiB0.07 GiB12±22%
Qwen3.5-27B-Deckard-PKD-Heretic-Uncensored-ThinkingI1-Q5_K_S27.4B17.40 GiB0.07 GiB18.53 GiB0.07 GiB12±22%
Qwen3.6-27B-Heretic2-ThinkingI1-Q5_K_S27.4B17.40 GiB0.07 GiB18.53 GiB0.07 GiB12±22%
Qwen3.6-27B-Uncensored-AggressiveI1-Q5_K_S27.4B17.40 GiB0.07 GiB18.53 GiB0.07 GiB12±22%
Qwen-3.5-Opus-GLM-27BI1-Q5_K_S26.9B17.40 GiB0.07 GiB18.53 GiB0.07 GiB12±22%
Qwen3.6-27B-abliteratedI1-Q5_K_S27.4B17.40 GiB0.07 GiB18.53 GiB0.07 GiB12±22%
KoQweopus-3.5-27B-experimentalI1-Q5_K_S27.8B17.40 GiB0.07 GiB18.53 GiB0.07 GiB12±22%
Webcoda-AI-27BI1-Q5_K_S27.4B17.40 GiB0.07 GiB18.53 GiB0.07 GiB12±22%
Qwen3.5-27B-imabari-v2I1-Q5_K_S27.8B17.40 GiB0.07 GiB18.53 GiB0.07 GiB12±22%
Qwen3.6-27B-AEON-Ultimate-Uncensored-BF16Q5_K_S27.4B17.40 GiB0.07 GiB18.53 GiB0.07 GiB12±22%
Huihui-Qwen3.5-27B-abliteratedI1-Q5_K_S27.8B17.40 GiB0.07 GiB18.53 GiB0.07 GiB12±22%
Qwen3.5-27B-uncensored-heretic-v1I1-Q5_K_S27.4B17.40 GiB0.07 GiB18.53 GiB0.07 GiB12±22%
Qwen3.5-27B-Unredacted-MAXI1-Q5_K_S27.4B17.40 GiB0.07 GiB18.53 GiB0.07 GiB12±22%
Qwen3.5-27B-hereticI1-Q5_K_S27.4B17.40 GiB0.07 GiB18.53 GiB0.07 GiB12±22%
Carnice-V2-27bI1-Q5_K_S27.4B17.40 GiB0.07 GiB18.53 GiB0.07 GiB12±22%
Qwen3.5-Queen-27BI1-Q5_K_S27.4B17.40 GiB0.07 GiB18.53 GiB0.07 GiB12±22%
GRaPE-2-ProI1-Q5_K_S27.8B17.40 GiB0.07 GiB18.53 GiB0.07 GiB12±22%
Huihui-Qwen3.6-27B-abliteratedQ5_K_S27.8B17.40 GiB0.07 GiB18.53 GiB0.07 GiB12±22%
Qwen3.5-27B-abliteratedQ5_K_S26.9B17.40 GiB0.07 GiB18.53 GiB0.07 GiB12±22%
Qwen3.5-27B-DerestrictedI1-Q5_K_S27.8B17.40 GiB0.07 GiB18.53 GiB0.07 GiB12±22%
ThinkingCap-Qwen3.6-27B-hereticQ5_K_S27.4B17.40 GiB0.07 GiB18.53 GiB0.07 GiB12±22%
●From the file◐Predictedwhat these mean

Speed is modeled, not measured: decode is memory-bandwidth bound, so tokens per second is bytes read per token against achievable bandwidth. Mixture-of-experts models carry a wider band because only the routed experts are read each step, and few have been measured publicly.

Questions people ask

What AI models can a RTX 4000 Ada Generation run?
1953 of 2118 indexed open-weight models fit a RTX 4000 Ada Generation at 4,096 context with q4_0 KV cache, the largest being EuroLLM-22B-Instruct-2512 at Q6_K. That covers text, vision-language, image, video and speech models.
How much usable memory does a RTX 4000 Ada Generation actually have?
Its nameplate is 20 GB, but about 18.60 GiB is available to a model once driver and compositor overhead is accounted for.
Is a RTX 4000 Ada Generation fast for local AI?
Its memory bandwidth is 360 GB/s, and that figure — not teraflops — is what governs token generation speed. Capacity decides what you can run; bandwidth decides how fast it runs.