NVIDIA · workstation

RTX 4000 Ada Generation

RTX 4000 Ada Generation has 20 GB of VRAM at 360 GB/s — about 18.60 GiB usable after driver and compositor overhead. 1944 of 2118 indexed models fit at 8K context with q8_0 KV.

Spec sheet· bandwidth, theoreticalFrom the file· fit from summed bytesPredicted· speed
Memory
20 GB
GDDR6
Bandwidth
360 GB/s
160-bit bus
Tensor FP16
107 TF
dense
TDP
130 W
$1250 MSRP
Context4K8K16K32K64K128K
KV cachef16q8_0q4_0quantizing the KV cache is a ~2× lever on the dominant term at long context
text 1669vision language 171video 16audio asr 39image 2audio tts 21embedding 26

What fits at 8K context

largest quantization that fits, per model · 1944 of 2118 indexed
ModelBest quantParamsWeights●KV●Total in memory◐Headroom◐tok/s◐
IQuest-Coder-V1-40B-InstructI1-IQ3_S39.8B16.17 GiB1.33 GiB18.60 GiB0.00 GiB12±22%
Gemma-4-Novelist-Eclipse-31BQ3_K_L32.7B16.22 GiB1.29 GiB18.59 GiB0.01 GiB12±22%
Gemma-4-31B-StyleTuneQ3_K_L32.7B16.22 GiB1.29 GiB18.59 GiB0.01 GiB12±22%
Qwen2.5-Coder-14B-InstructQ4_K_M14.8B16.74 GiB0.80 GiB18.58 GiB0.02 GiB12±22%
Qwen3.6-40B-Claude-4.6-Opus-Deckard-Heretic-Uncensored-ThinkingIQ3_M39.5B17.12 GiB0.40 GiB18.58 GiB0.02 GiB12±22%
Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16UD-IQ3_S33.0B17.53 GiB0.00 GiB18.57 GiB0.03 GiB12±22%
Seed-OSS-36B-Instruct-biprojected-norm-preserving-abliteratedI1-Q3_K_M36.2B16.41 GiB1.06 GiB18.57 GiB0.03 GiB12±22%
Seed-OSS-36B-InstructQ3_K_M36.2B16.41 GiB1.06 GiB18.57 GiB0.03 GiB12±22%
Hermes-4.3-36B-hereticI1-Q3_K_M36.2B16.41 GiB1.06 GiB18.57 GiB0.03 GiB12±22%
Hermes-4.3-36BQ3_K_M36.2B16.41 GiB1.06 GiB18.57 GiB0.03 GiB12±22%
Seed-OSS-36B-BaseQ3_K_M36.2B16.41 GiB1.06 GiB18.57 GiB0.03 GiB12±22%
granite-4.0-h-smallMoEQ4_032.2B17.51 GiB0.07 GiB18.56 GiB0.04 GiB34±37%
Skywork-R1V3-38BQ4_K_S38.4B17.49 GiB0.00 GiB18.56 GiB0.04 GiB12±22%
Qwen3-53B-A3B-2507-THINKING-TOTAL-RECALL-v2-MASTER-CODERMoEI1-Q2_K_S53.0B16.87 GiB0.70 GiB18.56 GiB0.04 GiB39±37%
dolphin-2.6-mixtral-8x7bMoEI1-IQ3_XXS46.7B16.99 GiB0.53 GiB18.56 GiB0.04 GiB21±37%
xLAM-8x7b-rMoEIQ3_XXS46.7B16.99 GiB0.53 GiB18.56 GiB0.04 GiB21±37%
NVIDIA-Nemotron-Nano-9B-v2BF168.9B16.57 GiB0.93 GiB18.54 GiB0.06 GiB12±22%
openNemo-9B-abliteratedBF168.9B16.57 GiB0.93 GiB18.54 GiB0.06 GiB12±22%
Qwen3-48B-A4B-Savant-Commander-Distill-12X-Closed-Open-Heretic-UncensoredMoEIQ4_XS33.6B16.93 GiB0.60 GiB18.54 GiB0.06 GiB27±37%
Gemma-3-27B-MeditronFOI1-Q4_128.8B16.81 GiB0.66 GiB18.54 GiB0.06 GiB12±22%
grug-27bQ4_K_L27.4B17.21 GiB0.27 GiB18.54 GiB0.06 GiB12±22%
Carnice-V2-27bQ4_K_L27.4B17.21 GiB0.27 GiB18.54 GiB0.06 GiB12±22%
Fara1.5-27BQ4_K_L27.4B17.21 GiB0.27 GiB18.54 GiB0.06 GiB12±22%
Marco-Mini-InstructMoEQ8_017.3B17.10 GiB0.46 GiB18.53 GiB0.07 GiB57±37%
Goetia-26B-A4B-v1.4MoEI1-Q5_K_S26.0B17.22 GiB0.32 GiB18.53 GiB0.07 GiB12±22%
G4-Moonlight-Dusk-26B-A4B-hereticMoEI1-Q5_K_S26.5B17.22 GiB0.32 GiB18.53 GiB0.07 GiB12±22%
Pantheon-Reasoning-26B-A4B-1.1-hereticMoEI1-Q5_K_S26.5B17.22 GiB0.32 GiB18.53 GiB0.07 GiB12±22%
G4-Moonlight-Dusk-26B-A4BMoEI1-Q5_K_S26.5B17.22 GiB0.32 GiB18.53 GiB0.07 GiB12±22%
Chimera-X-26B-A4BMoEI1-Q5_K_S26.5B17.22 GiB0.32 GiB18.53 GiB0.07 GiB12±22%
Pantheon-Reasoning-26B-A4B-1.1MoEI1-Q5_K_S26.5B17.22 GiB0.32 GiB18.53 GiB0.07 GiB12±22%
Gemma-4-26B-A4B-StyleTune-V2MoEI1-Q5_K_S26.5B17.22 GiB0.32 GiB18.53 GiB0.07 GiB12±22%
Gemma-4-26B-A4B-StyleTuneMoEI1-Q5_K_S26.5B17.22 GiB0.32 GiB18.53 GiB0.07 GiB12±22%
gemma-4-26b-a4b-heretic-styletune-v2-headMoEI1-Q5_K_S25.8B17.22 GiB0.32 GiB18.53 GiB0.07 GiB12±22%
Carnice-Qwen3.6-MoE-35B-A3BMoEI1-IQ4_XS36.0B17.44 GiB0.08 GiB18.53 GiB0.07 GiB69±37%
Qwen35B-Agent-R2-AbliteratedMoEI1-IQ4_XS34.7B17.44 GiB0.08 GiB18.53 GiB0.07 GiB69±37%
Qwen3.6-35B-A3B-Claude-4.6-Opus-Reasoning-DistilledMoEI1-IQ4_XS36.0B17.44 GiB0.08 GiB18.53 GiB0.07 GiB69±37%
Darwin-35B-A3B-OpusMoEI1-IQ4_XS36.0B17.44 GiB0.08 GiB18.53 GiB0.07 GiB69±37%
Qwen35B-Agent-R2MoEI1-IQ4_XS34.7B17.44 GiB0.08 GiB18.53 GiB0.07 GiB69±37%
Carnice-MoE-35B-A3BMoEI1-IQ4_XS36.0B17.44 GiB0.08 GiB18.53 GiB0.07 GiB69±37%
spoomplesmaxx-flash-35B-A3MoEI1-IQ4_XS35.1B17.44 GiB0.08 GiB18.53 GiB0.07 GiB69±37%
Huihui-Qwen3.6-35B-A3B-Claude-4.7-Opus-abliteratedMoEI1-IQ4_XS36.0B17.44 GiB0.08 GiB18.53 GiB0.07 GiB69±37%
Qwen3.6-35B-A3B-Uncensored-AggressiveMoEI1-IQ4_XS35.1B17.44 GiB0.08 GiB18.53 GiB0.07 GiB69±37%
WorldSim-Opus-3.6-35B-A3BMoEI1-IQ4_XS35.1B17.44 GiB0.08 GiB18.53 GiB0.07 GiB69±37%
Qwen3.6-35B-A3B-abliterated-MAXMoEI1-IQ4_XS35.1B17.44 GiB0.08 GiB18.53 GiB0.07 GiB69±37%
Huihui-Qwen3.6-35B-A3B-abliteratedMoEI1-IQ4_XS36.0B17.44 GiB0.08 GiB18.53 GiB0.07 GiB69±37%
Qwopus3.6-35B-A3B-v1MoEI1-IQ4_XS36.0B17.44 GiB0.08 GiB18.53 GiB0.07 GiB69±37%
Qwen3.6-35B-A3B-StyleTuneMoEI1-IQ4_XS35.1B17.44 GiB0.08 GiB18.53 GiB0.07 GiB69±37%
Qwen3.6-35B-A3B-abliteratedMoEI1-IQ4_XS35.1B17.44 GiB0.08 GiB18.53 GiB0.07 GiB69±37%
Qwen3.6-35B-A3B-abliterated-v4MoEIQ4_XS34.7B17.44 GiB0.08 GiB18.53 GiB0.07 GiB69±37%
0GM-1.0-35B-A3B-0427MoEI1-IQ4_XS36.0B17.44 GiB0.08 GiB18.53 GiB0.07 GiB69±37%
Qwen3.6-27B-A3B-CoderMoEI1-Q5_K_M26.7B17.44 GiB0.08 GiB18.53 GiB0.07 GiB57±37%
magnum-v2-32bIQ4_XS32.5B16.35 GiB1.06 GiB18.51 GiB0.09 GiB12±22%
GLM-4.7-Flash-hereticMoEQ4_K_M29.9B17.27 GiB0.22 GiB18.50 GiB0.10 GiB51±37%
Qwen3-14B-GPT-5.2-High-Reasoning-DistillQ4_K_M14.8B16.77 GiB0.66 GiB18.49 GiB0.11 GiB12±22%
Trinity-MiniMoEQ5_K_M26.1B17.36 GiB0.13 GiB18.48 GiB0.12 GiB52±37%
Wan2.2-Animate-14BQ8_017.3B17.43 GiB0.00 GiB18.47 GiB0.13 GiB12±22%
deepseek-coder-33b-instructQ3_K_L33.3B16.35 GiB1.03 GiB18.46 GiB0.14 GiB12±22%
deepseek-coder-33b-baseQ3_K_L33.3B16.35 GiB1.03 GiB18.46 GiB0.14 GiB12±22%
WhiteRabbitNeo-33B-v1Q3_K_L33.3B16.35 GiB1.03 GiB18.46 GiB0.14 GiB12±22%
spoomplesmaxx-v2.1-30BI1-Q4_K_M28.9B16.29 GiB1.06 GiB18.46 GiB0.14 GiB12±22%
●From the file◐Predictedwhat these mean

Speed is modeled, not measured: decode is memory-bandwidth bound, so tokens per second is bytes read per token against achievable bandwidth. Mixture-of-experts models carry a wider band because only the routed experts are read each step, and few have been measured publicly.

Questions people ask

What AI models can a RTX 4000 Ada Generation run?
1944 of 2118 indexed open-weight models fit a RTX 4000 Ada Generation at 8,192 context with q8_0 KV cache, the largest being IQuest-Coder-V1-40B-Instruct at I1-IQ3_S. That covers text, vision-language, image, video and speech models.
How much usable memory does a RTX 4000 Ada Generation actually have?
Its nameplate is 20 GB, but about 18.60 GiB is available to a model once driver and compositor overhead is accounted for.
Is a RTX 4000 Ada Generation fast for local AI?
Its memory bandwidth is 360 GB/s, and that figure — not teraflops — is what governs token generation speed. Capacity decides what you can run; bandwidth decides how fast it runs.