What AI models can a RTX 4000 Ada Generation run?

1930 of 2118 indexed open-weight models fit a RTX 4000 Ada Generation at 16,384 context with q8_0 KV cache, the largest being gemma-2-27b-it at Q4_K_M. That covers text, vision-language, image, video and speech models.

How much usable memory does a RTX 4000 Ada Generation actually have?

Its nameplate is 20 GB, but about 18.60 GiB is available to a model once driver and compositor overhead is accounted for.

Is a RTX 4000 Ada Generation fast for local AI?

Its memory bandwidth is 360 GB/s, and that figure — not teraflops — is what governs token generation speed. Capacity decides what you can run; bandwidth decides how fast it runs.

NVIDIA · workstation

RTX 4000 Ada Generation

RTX 4000 Ada Generation has 20 GB of VRAM at 360 GB/s — about 18.60 GiB usable after driver and compositor overhead. 1930 of 2118 indexed models fit at 16K context with q8_0 KV.

Spec sheet· bandwidth, theoreticalFrom the file· fit from summed bytesPredicted· speed

Memory

20 GB

GDDR6

Bandwidth

360 GB/s

160-bit bus

Tensor FP16

107 TF

dense

TDP

130 W

$1250 MSRP

Context4K 8K 16K 32K 64K 128K

KV cachef16 q8_0 q4_0quantizing the KV cache is a ~2× lever on the dominant term at long context

text 1655vision language 171video 16audio asr 39image 2audio tts 21embedding 26

What fits at 16K context

largest quantization that fits, per model · 1930 of 2118 indexed

Model	Best quant	Params	Weights●	KV●	Total in memory◐	Headroom◐	tok/s◐
gemma-2-27b-it	Q4_K_M	27.2B	15.50 GiB	1.96 GiB	18.59 GiB	0.01 GiB	12±22%
magnum-v4-27b	Q4_K_M	27.2B	15.50 GiB	1.96 GiB	18.59 GiB	0.01 GiB	12±22%
Yi-34B-200K-DARE-megamerge-v8	I1-Q3_K_M	34.4B	15.51 GiB	1.99 GiB	18.59 GiB	0.01 GiB	12±22%
dolphin-2.9.1-yi-1.5-34b-heretic	Q3_K_M	34.4B	15.51 GiB	1.99 GiB	18.59 GiB	0.01 GiB	12±22%
dolphin-2.9.1-yi-1.5-34b	I1-Q3_K_M	34.4B	15.51 GiB	1.99 GiB	18.59 GiB	0.01 GiB	12±22%
OrionStar-Yi-34B-Chat-Llama	I1-Q3_K_M	34.4B	15.51 GiB	1.99 GiB	18.59 GiB	0.01 GiB	12±22%
Yi-34B-200K-Llamafied	I1-Q3_K_M	34.4B	15.51 GiB	1.99 GiB	18.59 GiB	0.01 GiB	12±22%
Yi-1.5-34B	Q3_K_M	34.4B	15.51 GiB	1.99 GiB	18.59 GiB	0.01 GiB	12±22%
Merged-RP-Stew-V2-34B	I1-Q3_K_M	34.4B	15.51 GiB	1.99 GiB	18.59 GiB	0.01 GiB	12±22%
Capybara-Tess-Yi-34B-200K	I1-Q3_K_M	34.4B	15.51 GiB	1.99 GiB	18.59 GiB	0.01 GiB	12±22%
Seed-OSS-36B-Instruct-biprojected-norm-preserving-abliterated	I1-IQ3_M	36.2B	15.36 GiB	2.13 GiB	18.59 GiB	0.01 GiB	12±22%
Seed-OSS-36B-Instruct	IQ3_M	36.2B	15.36 GiB	2.13 GiB	18.59 GiB	0.01 GiB	12±22%
Hermes-4.3-36B-heretic	I1-IQ3_M	36.2B	15.36 GiB	2.13 GiB	18.59 GiB	0.01 GiB	12±22%
Hermes-4.3-36B	IQ3_M	36.2B	15.36 GiB	2.13 GiB	18.59 GiB	0.01 GiB	12±22%
spoomplesmaxx-v2.1-30B	I1-Q4_K_S	28.9B	15.35 GiB	2.13 GiB	18.59 GiB	0.01 GiB	12±22%
Huihui-granite-4.1-30b-abliterated	I1-Q4_K_S	28.9B	15.35 GiB	2.13 GiB	18.59 GiB	0.01 GiB	12±22%
granite-4.1-30b-heretic	I1-Q4_K_S	28.9B	15.35 GiB	2.13 GiB	18.59 GiB	0.01 GiB	12±22%
granite-4.1-30b	Q4_K_S	28.9B	15.35 GiB	2.13 GiB	18.59 GiB	0.01 GiB	12±22%
DeepSeek-R1-Distill-Llama-70B	UD-IQ1_S	70.6B	14.79 GiB	2.66 GiB	18.58 GiB	0.02 GiB	12±22%
Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16	UD-IQ3_S	33.0B	17.53 GiB	0.00 GiB	18.57 GiB	0.03 GiB	12±22%
Nous-Hermes-2-Yi-34B	Q3_K_M	34.4B	15.49 GiB	1.99 GiB	18.57 GiB	0.03 GiB	12±22%
Nous-Capybara-limarpv3-34B	Q3_K_M	34.4B	15.49 GiB	1.99 GiB	18.57 GiB	0.03 GiB	12±22%
Llama3.2-30B-A3B-II-Dark-Champion-INSTRUCT-Heretic-Abliterated-UncensoredMoE	I1-Q4_0	30.0B	16.00 GiB	1.56 GiB	18.57 GiB	0.03 GiB	26±37%
Salience-1.5-FlashMoE	Q4_K_S	31.1B	16.77 GiB	0.80 GiB	18.56 GiB	0.04 GiB	41±37%
Skywork-R1V3-38B	Q4_K_S	38.4B	17.49 GiB	0.00 GiB	18.56 GiB	0.04 GiB	12±22%
Trinity-MiniMoE	Q5_K_M	26.1B	17.36 GiB	0.20 GiB	18.55 GiB	0.05 GiB	51±37%
Hermes-4-70B	UD-IQ1_S	70.6B	14.77 GiB	2.66 GiB	18.55 GiB	0.05 GiB	12±22%
Llama-3.3-70B-Instruct	UD-IQ1_S	70.6B	14.77 GiB	2.66 GiB	18.55 GiB	0.05 GiB	12±22%
Qwen3.6-35B-A3B-uncensored-heretic-Native-MTP-PreservedMoE	Q3_K_L	35.1B	17.37 GiB	0.17 GiB	18.54 GiB	0.06 GiB	66±37%
Qwen3.5-35B-A3B-uncensored-heretic-v2-Native-MTP-PreservedMoE	Q3_K_L	35.1B	17.37 GiB	0.17 GiB	18.54 GiB	0.06 GiB	66±37%
Qwen3-VL-30B-A3B-ThinkingMoE	Q4_K_S	31.1B	16.75 GiB	0.80 GiB	18.54 GiB	0.06 GiB	41±37%
MiroThinker-v1.0-30BMoE	Q4_K_S	30.5B	16.75 GiB	0.80 GiB	18.54 GiB	0.06 GiB	41±37%
Qwen3-30B-A3BMoE	Q4_K_S	30.5B	16.75 GiB	0.80 GiB	18.54 GiB	0.06 GiB	41±37%
Qwen3-30B-A3B-Instruct-2507MoE	Q4_K_S	30.5B	16.75 GiB	0.80 GiB	18.54 GiB	0.06 GiB	41±37%
Qwen3-30B-A3B-Thinking-2507MoE	Q4_K_S	30.5B	16.75 GiB	0.80 GiB	18.54 GiB	0.06 GiB	41±37%
Pantheon-Proto-RP-1.8-30B-A3BMoE	Q4_K_S	30.5B	16.75 GiB	0.80 GiB	18.54 GiB	0.06 GiB	41±37%
Huihui-Qwen3.5-35B-A3B-abliteratedMoE	I1-IQ4_XS	36.0B	17.37 GiB	0.17 GiB	18.54 GiB	0.06 GiB	66±37%
Qwen3.5-35B-A3B-BaseMoE	I1-IQ4_XS	36.0B	17.37 GiB	0.17 GiB	18.54 GiB	0.06 GiB	66±37%
Qwen3.5-35B-A3B-Claude-4.6-Opus-Reasoning-DistilledMoE	I1-IQ4_XS	36.0B	17.37 GiB	0.17 GiB	18.54 GiB	0.06 GiB	66±37%
Tongyi-DeepResearch-30B-A3BMoE	Q4_K_S	30.5B	16.75 GiB	0.80 GiB	18.54 GiB	0.06 GiB	41±37%
TildeOpen-30B-Instruct-LV	I1-IQ4_XS	30.7B	15.46 GiB	1.99 GiB	18.53 GiB	0.07 GiB	12±22%
Gemma-4-Gembrain-X-Core-31B	I1-Q3_K_L	31.3B	15.49 GiB	1.95 GiB	18.52 GiB	0.08 GiB	12±22%
Gemma-4-Gembrain-X-31B	I1-Q3_K_L	31.3B	15.49 GiB	1.95 GiB	18.52 GiB	0.08 GiB	12±22%
Gemma-4-31B-Isometry-Fabled-Persona	I1-Q3_K_L	31.3B	15.49 GiB	1.95 GiB	18.52 GiB	0.08 GiB	12±22%
Versipellis-31B	I1-Q3_K_L	31.3B	15.49 GiB	1.95 GiB	18.52 GiB	0.08 GiB	12±22%
Gemma4-Gutenberg-31B	I1-Q3_K_L	31.3B	15.49 GiB	1.95 GiB	18.52 GiB	0.08 GiB	12±22%
G4-MeroMero-31B-uncensored-heretic	I1-Q3_K_L	31.3B	15.49 GiB	1.95 GiB	18.52 GiB	0.08 GiB	12±22%
Gemma-4-Novelist-31B	I1-Q3_K_L	31.3B	15.49 GiB	1.95 GiB	18.52 GiB	0.08 GiB	12±22%
Wanabi-Gemma4-31B	I1-Q3_K_L	31.3B	15.49 GiB	1.95 GiB	18.52 GiB	0.08 GiB	12±22%
G4-Alice-v1.2-31B	I1-Q3_K_L	31.3B	15.49 GiB	1.95 GiB	18.52 GiB	0.08 GiB	12±22%
Agares-31B-v1	I1-Q3_K_L	30.7B	15.49 GiB	1.95 GiB	18.52 GiB	0.08 GiB	12±22%
Gemma4-Gutenberg-31B-Heretic	I1-Q3_K_L	31.3B	15.49 GiB	1.95 GiB	18.52 GiB	0.08 GiB	12±22%
gemma-4-Ortenzya-The-Creative-Wordsmith-31B-it-uncensored-heretic	I1-Q3_K_L	31.3B	15.49 GiB	1.95 GiB	18.52 GiB	0.08 GiB	12±22%
Gemma-4-Gemsicle-31B	I1-Q3_K_L	31.3B	15.49 GiB	1.95 GiB	18.52 GiB	0.08 GiB	12±22%
Gemma-4-Gembrain-31B-it-uncensored-heretic	I1-Q3_K_L	31.3B	15.49 GiB	1.95 GiB	18.52 GiB	0.08 GiB	12±22%
Melinoe-Gemma4-31B-VL-heretic	I1-Q3_K_L	31.3B	15.49 GiB	1.95 GiB	18.52 GiB	0.08 GiB	12±22%
G4-MeroMero-31B	I1-Q3_K_L	31.3B	15.49 GiB	1.95 GiB	18.52 GiB	0.08 GiB	12±22%
Glistening-Gem-31B-v1.0	I1-Q3_K_L	31.3B	15.49 GiB	1.95 GiB	18.52 GiB	0.08 GiB	12±22%
Melinoe-Gemma4-31B-VL	I1-Q3_K_L	31.3B	15.49 GiB	1.95 GiB	18.52 GiB	0.08 GiB	12±22%
Gemma-4-31B-Storymaxxed3	I1-Q3_K_L	31.3B	15.49 GiB	1.95 GiB	18.52 GiB	0.08 GiB	12±22%

●From the file◐Predictedwhat these mean

Speed is modeled, not measured: decode is memory-bandwidth bound, so tokens per second is bytes read per token against achievable bandwidth. Mixture-of-experts models carry a wider band because only the routed experts are read each step, and few have been measured publicly.

Questions people ask

What AI models can a RTX 4000 Ada Generation run?: 1930 of 2118 indexed open-weight models fit a RTX 4000 Ada Generation at 16,384 context with q8_0 KV cache, the largest being gemma-2-27b-it at Q4_K_M. That covers text, vision-language, image, video and speech models.
How much usable memory does a RTX 4000 Ada Generation actually have?: Its nameplate is 20 GB, but about 18.60 GiB is available to a model once driver and compositor overhead is accounted for.
Is a RTX 4000 Ada Generation fast for local AI?: Its memory bandwidth is 360 GB/s, and that figure — not teraflops — is what governs token generation speed. Capacity decides what you can run; bandwidth decides how fast it runs.