NVIDIA · workstation

RTX 4500 Ada Generation

RTX 4500 Ada Generation has 24 GB of VRAM at 432 GB/s — about 22.32 GiB usable after driver and compositor overhead. 1966 of 2118 indexed models fit at 8K context with q4_0 KV.

Spec sheet· bandwidth, theoreticalFrom the file· fit from summed bytesPredicted· speed
Memory
24 GB
GDDR6
Bandwidth
432 GB/s
192-bit bus
Tensor FP16
159 TF
dense
TDP
210 W
$2250 MSRP
Context4K8K16K32K64K128K
KV cachef16q8_0q4_0quantizing the KV cache is a ~2× lever on the dominant term at long context
text 1689vision language 173image 2video 16audio tts 21audio asr 39embedding 26

What fits at 8K context

largest quantization that fits, per model · 1966 of 2118 indexed
ModelBest quantParamsWeights●KV●Total in memory◐Headroom◐tok/s◐
dolphin-2.6-mixtral-8x7bMoEI1-Q3_K_M46.7B21.00 GiB0.28 GiB22.32 GiB0.00 GiB22±37%
Nous-Hermes-2-Mixtral-8x7B-DPOMoEQ3_K_M46.7B21.00 GiB0.28 GiB22.32 GiB0.00 GiB22±37%
Mixtral-8x7B-Instruct-v0.1MoEQ3_K_M46.7B21.00 GiB0.28 GiB22.32 GiB0.00 GiB22±37%
xLAM-8x7b-rMoEQ3_K_M46.7B21.00 GiB0.28 GiB22.32 GiB0.00 GiB22±37%
Skyfall-31B-v4.2-hereticI1-Q5_K_M31.4B20.72 GiB0.47 GiB22.32 GiB0.00 GiB12±22%
Skyfall-31B-v4.2I1-Q5_K_M31.4B20.72 GiB0.47 GiB22.32 GiB0.00 GiB12±22%
dolphin-2.5-mixtral-8x7bMoEQ3_K_M46.7B21.00 GiB0.28 GiB22.31 GiB0.01 GiB22±37%
Mixtral-8x7B-v0.1MoEQ3_K_M46.7B21.00 GiB0.28 GiB22.31 GiB0.01 GiB22±37%
L3-DARKEST-PLANET-16.5BQ2_K16.5B20.65 GiB0.62 GiB22.31 GiB0.01 GiB12±22%
HarmonicHarlequin_v5-20BIQ4_XS33.3B16.67 GiB4.57 GiB22.28 GiB0.04 GiB12±22%
Fallen-Gemma3-27B-v1Q6_K_L27.4B20.96 GiB0.26 GiB22.26 GiB0.06 GiB12±22%
diffusiongemma-26B-A4B-it-HERETIC-UncensoredMoEQ6_K25.8B21.10 GiB0.17 GiB22.26 GiB0.06 GiB12±22%
diffusiongemma-26B-A4B-itMoEQ6_K25.8B21.10 GiB0.17 GiB22.26 GiB0.06 GiB12±22%
Open_Gpt4_8x7B_v0.2MoEQ3_K_M46.7B20.93 GiB0.28 GiB22.25 GiB0.07 GiB22±37%
Goetia-26B-A4B-v1.3-Absolute-Heretic-ARAMoEI1-Q6_K25.8B21.08 GiB0.17 GiB22.24 GiB0.08 GiB12±22%
Frank-26B-A4BMoEI1-Q6_K26.5B21.08 GiB0.17 GiB22.24 GiB0.08 GiB12±22%
G4-MeroMero-26B-A4B-it-uncensored-hereticMoEI1-Q6_K25.8B21.08 GiB0.17 GiB22.24 GiB0.08 GiB12±22%
EVE-26b-XENO-HATMoEI1-Q6_K25.8B21.08 GiB0.17 GiB22.24 GiB0.08 GiB12±22%
Gemma-4-26B-A4B-Animus-V14.1-FFT-hereticMoEI1-Q6_K25.8B21.08 GiB0.17 GiB22.24 GiB0.08 GiB12±22%
gemma-4-26B-A4B-it-Claude-Opus-DistillMoEQ6_K26.5B21.08 GiB0.17 GiB22.24 GiB0.08 GiB12±22%
G4-MeroMero-26B-A4BMoEI1-Q6_K25.8B21.08 GiB0.17 GiB22.24 GiB0.08 GiB12±22%
gemma-4-26B-A4B-it-Claude-Opus-Distill-v2MoEQ6_K26.5B21.08 GiB0.17 GiB22.24 GiB0.08 GiB12±22%
G4-Dark-Soul-26B-A4BMoEI1-Q6_K25.8B21.08 GiB0.17 GiB22.24 GiB0.08 GiB12±22%
gemma-4-26B-A4B-it-local-abliterated-sota-internal-t34MoEI1-Q6_K25.8B21.08 GiB0.17 GiB22.24 GiB0.08 GiB12±22%
gemma-4-26B-A4B-it-SOMPOA-heresyMoEI1-Q6_K25.8B21.08 GiB0.17 GiB22.24 GiB0.08 GiB12±22%
gemma-4-26B-A4B-it-hereticMoEI1-Q6_K25.8B21.08 GiB0.17 GiB22.24 GiB0.08 GiB12±22%
gemma-4-26B-A4B-it-abliterixMoEI1-Q6_K25.8B21.08 GiB0.17 GiB22.24 GiB0.08 GiB12±22%
gemma-4-26B-A4B-it-heretic-ara-v2MoEI1-Q6_K25.8B21.08 GiB0.17 GiB22.24 GiB0.08 GiB12±22%
Gemma-4-26B-A4B-it-heretic-antislopMoEI1-Q6_K25.8B21.08 GiB0.17 GiB22.24 GiB0.08 GiB12±22%
gemma-4-26B-A4B-it-ultra-uncensored-hereticMoEQ6_K25.8B21.08 GiB0.17 GiB22.24 GiB0.08 GiB12±22%
gemma-4-26B-A4B-it-uncensored-hereticMoEQ6_K25.8B21.08 GiB0.17 GiB22.24 GiB0.08 GiB12±22%
gemma-4-26B-A4B-Heretic-StableMoEI1-Q6_K25.8B21.08 GiB0.17 GiB22.24 GiB0.08 GiB12±22%
gemma-4-26B-A4B-it-Uncensored-MAXMoEI1-Q6_K25.8B21.08 GiB0.17 GiB22.24 GiB0.08 GiB12±22%
gemma-4-26B-A4B-it-ara-abliteratedMoEI1-Q6_K25.8B21.08 GiB0.17 GiB22.24 GiB0.08 GiB12±22%
Huihui-gemma-4-26B-A4B-it-abliteratedMoEI1-Q6_K26.5B21.08 GiB0.17 GiB22.24 GiB0.08 GiB12±22%
Gemma-4-26B-A4B-AbliteratedMoEI1-Q6_K25.8B21.08 GiB0.17 GiB22.24 GiB0.08 GiB12±22%
gemma4-26b-fiction-bf16MoEI1-Q6_K25.8B21.08 GiB0.17 GiB22.24 GiB0.08 GiB12±22%
gemma-4-26B-A4B-it-heretic-araMoEI1-Q6_K25.8B21.08 GiB0.17 GiB22.24 GiB0.08 GiB12±22%
gemma-4-26B-A4B-it-abliteratedMoEQ6_K25.8B21.08 GiB0.17 GiB22.24 GiB0.08 GiB12±22%
gemma-4-26B-A4BMoEQ6_K26.5B21.08 GiB0.17 GiB22.24 GiB0.08 GiB12±22%
Hunyuan-A13B-InstructMoEUD-TQ1_080.4B20.95 GiB0.28 GiB22.23 GiB0.09 GiB12±22%
umt5-xxlF325.7B21.17 GiB0.00 GiB22.22 GiB0.10 GiB12±22%
Qwen3.5-40B-RoughHouse-Claude-4.6-Opus-Polar-Deckard-Uncensored-Heretic-ThinkingI1-Q4_K_S39.5B20.94 GiB0.21 GiB22.21 GiB0.11 GiB12±22%
GLM-4-32B-0414-Korean-CultureI1-Q5_K_S32.6B20.98 GiB0.13 GiB22.20 GiB0.12 GiB12±22%
GLM-Z1-32B-0414Q5_K_S32.6B20.98 GiB0.13 GiB22.20 GiB0.12 GiB12±22%
GLM-4-32B-0414Q5_K_S32.6B20.98 GiB0.13 GiB22.20 GiB0.12 GiB12±22%
GLM-Z1-32B-0414-uncensored-heretic-v2Q5_K_S32.6B20.98 GiB0.13 GiB22.20 GiB0.12 GiB12±22%
Qwen3.5-40B-Claude-4.6-Opus-Deckard-Heretic-Uncensored-ThinkingI1-Q4_K_S39.5B20.92 GiB0.21 GiB22.20 GiB0.12 GiB12±22%
Olmo-3.1-32B-InstructQ5_K_S32.2B20.71 GiB0.38 GiB22.18 GiB0.14 GiB12±22%
Olmo-3.1-32B-ThinkQ5_K_S32.2B20.71 GiB0.38 GiB22.18 GiB0.14 GiB12±22%
Olmo-3-32B-ThinkQ5_K_S32.2B20.71 GiB0.38 GiB22.18 GiB0.14 GiB12±22%
Kimi-Linear-48B-A3B-InstructMoEIQ3_M49.1B21.10 GiB0.07 GiB22.17 GiB0.15 GiB12±22%
ALIA-40b-fc-2606I1-IQ4_XS40.4B20.64 GiB0.42 GiB22.17 GiB0.15 GiB12±22%
ALIA-40b-instruct-2606I1-IQ4_XS40.4B20.64 GiB0.42 GiB22.17 GiB0.15 GiB12±22%
Qwen3.6-35B-A3BMoEUD-Q4_K_M36.0B21.11 GiB0.04 GiB22.16 GiB0.16 GiB71±37%
Qwen3.5-35B-A3BMoEQ4_K_L36.0B21.11 GiB0.04 GiB22.15 GiB0.17 GiB71±37%
Skywork-R1V3-38BQ5_K_S38.4B21.08 GiB0.00 GiB22.15 GiB0.17 GiB12±22%
Gemma-4-Gembrain-X-Core-31BI1-Q5_K_M31.3B20.35 GiB0.68 GiB22.11 GiB0.21 GiB12±22%
Gemma-4-Gembrain-X-31BI1-Q5_K_M31.3B20.35 GiB0.68 GiB22.11 GiB0.21 GiB12±22%
Gemma-4-31B-Isometry-Fabled-PersonaI1-Q5_K_M31.3B20.35 GiB0.68 GiB22.11 GiB0.21 GiB12±22%
●From the file◐Predictedwhat these mean

Speed is modeled, not measured: decode is memory-bandwidth bound, so tokens per second is bytes read per token against achievable bandwidth. Mixture-of-experts models carry a wider band because only the routed experts are read each step, and few have been measured publicly.

Questions people ask

What AI models can a RTX 4500 Ada Generation run?
1966 of 2118 indexed open-weight models fit a RTX 4500 Ada Generation at 8,192 context with q4_0 KV cache, the largest being dolphin-2.6-mixtral-8x7b at I1-Q3_K_M. That covers text, vision-language, image, video and speech models.
How much usable memory does a RTX 4500 Ada Generation actually have?
Its nameplate is 24 GB, but about 22.32 GiB is available to a model once driver and compositor overhead is accounted for.
Is a RTX 4500 Ada Generation fast for local AI?
Its memory bandwidth is 432 GB/s, and that figure — not teraflops — is what governs token generation speed. Capacity decides what you can run; bandwidth decides how fast it runs.