NVIDIA · workstation

RTX 4500 Ada Generation

RTX 4500 Ada Generation has 24 GB of VRAM at 432 GB/s — about 22.32 GiB usable after driver and compositor overhead. 1871 of 2118 indexed models fit at 64K context with q8_0 KV.

Spec sheet· bandwidth, theoreticalFrom the file· fit from summed bytesPredicted· speed
Memory
24 GB
GDDR6
Bandwidth
432 GB/s
192-bit bus
Tensor FP16
159 TF
dense
TDP
210 W
$2250 MSRP
Context4K8K16K32K64K128K
KV cachef16q8_0q4_0quantizing the KV cache is a ~2× lever on the dominant term at long context
text 1596vision language 171image 2audio tts 21audio asr 39video 16embedding 26

What fits at 64K context

largest quantization that fits, per model · 1871 of 2118 indexed
ModelBest quantParamsWeights●KV●Total in memory◐Headroom◐tok/s◐
Yi-34B-200K-DARE-megamerge-v8I1-IQ3_XS34.4B13.26 GiB7.97 GiB22.31 GiB0.01 GiB12±22%
dolphin-2.9.1-yi-1.5-34bI1-IQ3_XS34.4B13.26 GiB7.97 GiB22.31 GiB0.01 GiB12±22%
OrionStar-Yi-34B-Chat-LlamaI1-IQ3_XS34.4B13.26 GiB7.97 GiB22.31 GiB0.01 GiB12±22%
Yi-34B-200K-LlamafiedI1-IQ3_XS34.4B13.26 GiB7.97 GiB22.31 GiB0.01 GiB12±22%
Yi-1.5-34BIQ3_XS34.4B13.26 GiB7.97 GiB22.31 GiB0.01 GiB12±22%
Merged-RP-Stew-V2-34BI1-IQ3_XS34.4B13.26 GiB7.97 GiB22.31 GiB0.01 GiB12±22%
Capybara-Tess-Yi-34B-200KI1-IQ3_XS34.4B13.26 GiB7.97 GiB22.31 GiB0.01 GiB12±22%
Nous-Capybara-limarpv3-34BI1-IQ3_XXS34.4B13.24 GiB7.97 GiB22.29 GiB0.03 GiB12±22%
grug-27bQ5_K_M27.4B19.10 GiB2.13 GiB22.29 GiB0.03 GiB12±22%
Carnice-V2-27bQ5_K_M27.4B19.10 GiB2.13 GiB22.29 GiB0.03 GiB12±22%
Fara1.5-27BQ5_K_M27.4B19.10 GiB2.13 GiB22.29 GiB0.03 GiB12±22%
v6-Finch-7B-HFIQ4_XS7.6B4.24 GiB17.00 GiB22.28 GiB0.04 GiB12±22%
rwkv-6-world-7bIQ4_XS7.6B4.24 GiB17.00 GiB22.28 GiB0.04 GiB12±22%
Qwen-AgentWorld-35B-A3BMoEUD-Q4_K_M34.7B20.61 GiB0.66 GiB22.28 GiB0.04 GiB55±37%
Ornith-1.0-35BMoEUD-Q4_K_M34.7B20.61 GiB0.66 GiB22.28 GiB0.04 GiB55±37%
Le-Chaton-Slim-23BMoEI1-Q6_K23.3B17.81 GiB3.45 GiB22.28 GiB0.04 GiB18±37%
magnum-v4-27bQ4_027.2B14.60 GiB6.54 GiB22.28 GiB0.04 GiB12±22%
dolphin-2.6-mixtral-8x7bMoEI1-IQ3_XXS46.7B16.99 GiB4.25 GiB22.28 GiB0.04 GiB15±37%
xLAM-8x7b-rMoEIQ3_XXS46.7B16.99 GiB4.25 GiB22.28 GiB0.04 GiB15±37%
Seed-OSS-36B-Instruct-biprojected-norm-preserving-abliteratedI1-Q2_K36.2B12.67 GiB8.50 GiB22.27 GiB0.05 GiB12±22%
Seed-OSS-36B-InstructQ2_K36.2B12.67 GiB8.50 GiB22.27 GiB0.05 GiB12±22%
Hermes-4.3-36B-hereticI1-Q2_K36.2B12.67 GiB8.50 GiB22.27 GiB0.05 GiB12±22%
Hermes-4.3-36BQ2_K36.2B12.67 GiB8.50 GiB22.27 GiB0.05 GiB12±22%
magnum-v2-32bIQ3_XS32.5B12.67 GiB8.50 GiB22.27 GiB0.05 GiB12±22%
Nex-N2-miniMoEUD-Q4_K_M35.1B20.60 GiB0.66 GiB22.26 GiB0.06 GiB55±37%
GLM-4-32B-0414-Korean-CultureI1-Q4_132.6B19.14 GiB2.03 GiB22.25 GiB0.07 GiB12±22%
GLM-Z1-32B-0414Q4_132.6B19.14 GiB2.03 GiB22.25 GiB0.07 GiB12±22%
GLM-4-32B-0414Q4_132.6B19.14 GiB2.03 GiB22.25 GiB0.07 GiB12±22%
deepseek-math-7b-instructQ6_K6.9B5.28 GiB15.94 GiB22.25 GiB0.07 GiB12±22%
deepseek-llm-7b-chatQ6_K6.9B5.28 GiB15.94 GiB22.25 GiB0.07 GiB12±22%
Janus-Pro-7BI1-Q6_K7.4B5.28 GiB15.94 GiB22.25 GiB0.07 GiB12±22%
deepseek-coder-7b-instruct-v1.5I1-Q6_K6.9B5.28 GiB15.94 GiB22.25 GiB0.07 GiB12±22%
GLM-4.7-Flash-hereticMoEQ5_K_S29.9B19.48 GiB1.76 GiB22.24 GiB0.08 GiB34±37%
granite-4.0-h-smallMoEQ5_032.2B20.72 GiB0.53 GiB22.24 GiB0.08 GiB31±37%
gemma-2-27b-itIQ4_NL27.2B14.56 GiB6.54 GiB22.23 GiB0.09 GiB12±22%
Magistry-24B-v1.1Q5_K_M23.6B15.79 GiB5.31 GiB22.22 GiB0.10 GiB12±22%
umt5-xxlF325.7B21.17 GiB0.00 GiB22.22 GiB0.10 GiB12±22%
Ornith-Agents-A1-3.7-35B-A3B-dare_ties_v4MoEQ4_K_M34.7B20.55 GiB0.66 GiB22.22 GiB0.10 GiB55±37%
Ornith-Agents-A1-3.6-35B-A3B-dare_tiesMoEQ4_K_M34.7B20.55 GiB0.66 GiB22.22 GiB0.10 GiB55±37%
phi-4Q8_014.7B14.51 GiB6.64 GiB22.21 GiB0.11 GiB12±22%
Phi-4-reasoningQ8_014.7B14.51 GiB6.64 GiB22.21 GiB0.11 GiB12±22%
Phi-4-reasoning-plusQ8_014.7B14.51 GiB6.64 GiB22.21 GiB0.11 GiB12±22%
OLMo-2-1124-7B-InstructQ4_K_M7.3B4.16 GiB17.00 GiB22.19 GiB0.13 GiB12±22%
Qwen3.6-35B-A3BMoEQ4_036.0B20.51 GiB0.66 GiB22.18 GiB0.14 GiB55±37%
gemma-4-A4B-98e-v6-coder-itMoEQ8_020.5B19.71 GiB1.48 GiB22.18 GiB0.14 GiB12±22%
gemma-4-A4B-98e-v7-coder-itMoEQ8_020.5B19.71 GiB1.48 GiB22.18 GiB0.14 GiB12±22%
gemma-4-A4B-98e-v7-coderx-itMoEQ8_020.5B19.71 GiB1.48 GiB22.18 GiB0.14 GiB12±22%
Olmo-3.1-32B-InstructQ4_K_L32.2B18.50 GiB2.57 GiB22.17 GiB0.15 GiB12±22%
Olmo-3.1-32B-ThinkQ4_K_L32.2B18.50 GiB2.57 GiB22.17 GiB0.15 GiB12±22%
Olmo-3-32B-ThinkQ4_K_L32.2B18.50 GiB2.57 GiB22.17 GiB0.15 GiB12±22%
gemma-4-26B-A4B-itMoEUD-Q5_K_M26.5B19.70 GiB1.48 GiB22.17 GiB0.15 GiB12±22%
Huihui-gemma-4-26B-A4B-it-abliteratedMoEUD-Q5_K_M26.5B19.70 GiB1.48 GiB22.17 GiB0.15 GiB12±22%
GLM-Z1-Rumination-32B-0414IQ3_XS33.1B12.98 GiB8.10 GiB22.17 GiB0.15 GiB12±22%
GLM-4.7-FlashMoEQ5_K_S31.2B19.39 GiB1.76 GiB22.15 GiB0.17 GiB34±37%
Qwen3-Coder-NextMoEIQ2_XXS79.7B17.97 GiB3.19 GiB22.15 GiB0.17 GiB30±37%
Qwen3-Next-80B-A3B-ThinkingMoEIQ2_XXS81.3B17.97 GiB3.19 GiB22.15 GiB0.17 GiB30±37%
Qwen3-Next-80B-A3B-InstructMoEIQ2_XXS81.3B17.97 GiB3.19 GiB22.15 GiB0.17 GiB30±37%
Skywork-R1V3-38BQ5_K_S38.4B21.08 GiB0.00 GiB22.15 GiB0.17 GiB12±22%
North-Mini-Code-1.0MoEUD-Q5_K_S30.5B20.13 GiB1.03 GiB22.14 GiB0.18 GiB41±37%
Kimi-Linear-48B-A3B-InstructMoEQ3_K_S49.1B20.12 GiB1.01 GiB22.14 GiB0.18 GiB12±22%
●From the file◐Predictedwhat these mean

Speed is modeled, not measured: decode is memory-bandwidth bound, so tokens per second is bytes read per token against achievable bandwidth. Mixture-of-experts models carry a wider band because only the routed experts are read each step, and few have been measured publicly.

Questions people ask

What AI models can a RTX 4500 Ada Generation run?
1871 of 2118 indexed open-weight models fit a RTX 4500 Ada Generation at 65,536 context with q8_0 KV cache, the largest being Yi-34B-200K-DARE-megamerge-v8 at I1-IQ3_XS. That covers text, vision-language, image, video and speech models.
How much usable memory does a RTX 4500 Ada Generation actually have?
Its nameplate is 24 GB, but about 22.32 GiB is available to a model once driver and compositor overhead is accounted for.
Is a RTX 4500 Ada Generation fast for local AI?
Its memory bandwidth is 432 GB/s, and that figure — not teraflops — is what governs token generation speed. Capacity decides what you can run; bandwidth decides how fast it runs.