What AI models can a RTX 4500 Ada Generation run?

1960 of 2118 indexed open-weight models fit a RTX 4500 Ada Generation at 16,384 context with q4_0 KV cache, the largest being Delphi-25B-SimpleRL-Math at I1-Q5_K_M. That covers text, vision-language, image, video and speech models.

How much usable memory does a RTX 4500 Ada Generation actually have?

Its nameplate is 24 GB, but about 22.32 GiB is available to a model once driver and compositor overhead is accounted for.

Is a RTX 4500 Ada Generation fast for local AI?

Its memory bandwidth is 432 GB/s, and that figure — not teraflops — is what governs token generation speed. Capacity decides what you can run; bandwidth decides how fast it runs.

NVIDIA · workstation

RTX 4500 Ada Generation

RTX 4500 Ada Generation has 24 GB of VRAM at 432 GB/s — about 22.32 GiB usable after driver and compositor overhead. 1960 of 2118 indexed models fit at 16K context with q4_0 KV.

Spec sheet· bandwidth, theoreticalFrom the file· fit from summed bytesPredicted· speed

Memory

24 GB

GDDR6

Bandwidth

432 GB/s

192-bit bus

Tensor FP16

159 TF

dense

TDP

210 W

$2250 MSRP

Context4K 8K 16K 32K 64K 128K

KV cachef16 q8_0 q4_0quantizing the KV cache is a ~2× lever on the dominant term at long context

text 1683vision language 173image 2video 16audio tts 21audio asr 39embedding 26

What fits at 16K context

largest quantization that fits, per model · 1960 of 2118 indexed

Model	Best quant	Params	Weights●	KV●	Total in memory◐	Headroom◐	tok/s◐
Delphi-25B-SimpleRL-Math	I1-Q5_K_M	25.0B	16.52 GiB	4.71 GiB	22.31 GiB	0.01 GiB	12±22%
Skyfall-31B-v4.2	Q5_K_S	31.4B	20.23 GiB	0.95 GiB	22.30 GiB	0.02 GiB	12±22%
command-r-35b-writer-v2	I1-IQ3_M	35.0B	15.55 GiB	5.63 GiB	22.28 GiB	0.04 GiB	12±22%
medgemma-27b-it	I1-Q6_K	28.8B	20.64 GiB	0.52 GiB	22.25 GiB	0.07 GiB	12±22%
gemma-3-27b-it-abliterated-refined-vision	I1-Q6_K	27.4B	20.64 GiB	0.52 GiB	22.25 GiB	0.07 GiB	12±22%
gemma-3-27b-it-abliterated	Q6_K	27.4B	20.64 GiB	0.52 GiB	22.25 GiB	0.07 GiB	12±22%
Nidum-Gemma-3-27B-it-Uncensored	I1-Q6_K	27.4B	20.64 GiB	0.52 GiB	22.25 GiB	0.07 GiB	12±22%
gemma-3-27b-it	Q6_K	27.4B	20.64 GiB	0.52 GiB	22.25 GiB	0.07 GiB	12±22%
AtomicGPT-gemma3-27b	I1-Q6_K	27.4B	20.64 GiB	0.52 GiB	22.25 GiB	0.07 GiB	12±22%
Unbound-v1.12.0-27B	I1-Q6_K	27.4B	20.64 GiB	0.52 GiB	22.25 GiB	0.07 GiB	12±22%
Mira-v1.12-Ties-27B	I1-Q6_K	27.4B	20.64 GiB	0.52 GiB	22.25 GiB	0.07 GiB	12±22%
Medgamma27B	I1-Q6_K	27.0B	20.64 GiB	0.52 GiB	22.25 GiB	0.07 GiB	12±22%
medgemma-27b-text-it	Q6_K	27.0B	20.64 GiB	0.52 GiB	22.25 GiB	0.07 GiB	12±22%
Kimi-Linear-48B-A3B-InstructMoE	IQ3_M	49.1B	21.10 GiB	0.13 GiB	22.24 GiB	0.08 GiB	12±22%
Pantheon-Reasoning-27B	I1-Q6_K	27.8B	20.89 GiB	0.28 GiB	22.23 GiB	0.09 GiB	12±22%
Qwen3.6-27B-uncensored-heretic-v2-Native-MTP-Preserved	I1-Q6_K	27.4B	20.89 GiB	0.28 GiB	22.23 GiB	0.09 GiB	12±22%
Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP	I1-Q6_K	27.8B	20.89 GiB	0.28 GiB	22.23 GiB	0.09 GiB	12±22%
Qwen3.6-27B-Fable-5-Experimental	I1-Q6_K	27.8B	20.89 GiB	0.28 GiB	22.23 GiB	0.09 GiB	12±22%
Qwable-5-27B-Coder	I1-Q6_K	27.8B	20.89 GiB	0.28 GiB	22.23 GiB	0.09 GiB	12±22%
Qwen3.6-27B-AEON-Ultimate-Uncensored-BF16	Q6_K	27.4B	20.89 GiB	0.28 GiB	22.23 GiB	0.09 GiB	12±22%
EVE-27b-XENO-HAT-DeepSeek-V4-Flash	I1-Q6_K	27.8B	20.89 GiB	0.28 GiB	22.23 GiB	0.09 GiB	12±22%
EVE-27B-XENO-HAT	I1-Q6_K	27.8B	20.89 GiB	0.28 GiB	22.23 GiB	0.09 GiB	12±22%
Godoter-27B	I1-Q6_K	27.8B	20.89 GiB	0.28 GiB	22.23 GiB	0.09 GiB	12±22%
Reasoning-Medical-27B	I1-Q6_K	27.8B	20.89 GiB	0.28 GiB	22.23 GiB	0.09 GiB	12±22%
Qwopus3.6-27B-v2-abliterated	I1-Q6_K	27.4B	20.89 GiB	0.28 GiB	22.23 GiB	0.09 GiB	12±22%
Qwen3.6-27B-Uncensored-HauhauCS-Aggressive-BF16	I1-Q6_K	27.8B	20.89 GiB	0.28 GiB	22.23 GiB	0.09 GiB	12±22%
Reasoning-Medical0.1-27B	I1-Q6_K	27.8B	20.89 GiB	0.28 GiB	22.23 GiB	0.09 GiB	12±22%
Huihui-ThinkingCap-Qwen3.6-27B-abliterated	I1-Q6_K	27.4B	20.89 GiB	0.28 GiB	22.23 GiB	0.09 GiB	12±22%
Semancer-27B	I1-Q6_K	27.8B	20.89 GiB	0.28 GiB	22.23 GiB	0.09 GiB	12±22%
Qwen3.6-27B-Uncensored-Cyber	Q6_K	27.4B	20.89 GiB	0.28 GiB	22.23 GiB	0.09 GiB	12±22%
Qwen3.6-27B-Omnimerge-v4	Q6_K	27.8B	20.89 GiB	0.28 GiB	22.23 GiB	0.09 GiB	12±22%
Qwopus3.6-27B-v2	Q6_K	27.8B	20.89 GiB	0.28 GiB	22.23 GiB	0.09 GiB	12±22%
Darwin-28B-Coder	I1-Q6_K	26.9B	20.89 GiB	0.28 GiB	22.23 GiB	0.09 GiB	12±22%
Qwopus3.6-27B-Coder	Q6_K	27.8B	20.89 GiB	0.28 GiB	22.23 GiB	0.09 GiB	12±22%
Skyfall-31B-v4.2-heretic	I1-Q5_K_S	31.4B	20.17 GiB	0.95 GiB	22.23 GiB	0.09 GiB	12±22%
Maenad-70B	I1-IQ2_XS	70.6B	19.69 GiB	1.41 GiB	22.22 GiB	0.10 GiB	12±22%
DeepSeek-R1-Distill-Llama-70B-Uncensored-v2-Unbiased-Reasoner	I1-IQ2_XS	70.6B	19.69 GiB	1.41 GiB	22.22 GiB	0.10 GiB	12±22%
Rombos-LLM-70b-Llama-3.3	I1-IQ2_XS	70.6B	19.69 GiB	1.41 GiB	22.22 GiB	0.10 GiB	12±22%
L3.3-Electra-R1-70b	I1-IQ2_XS	70.6B	19.69 GiB	1.41 GiB	22.22 GiB	0.10 GiB	12±22%
L3.3-70B-Magnum-v4-SE	IQ2_XS	70.6B	19.69 GiB	1.41 GiB	22.22 GiB	0.10 GiB	12±22%
Latxa-Llama-3.1-70B-Instruct-v2	I1-IQ2_XS	70.6B	19.69 GiB	1.41 GiB	22.22 GiB	0.10 GiB	12±22%
Llama-3.3_70_b_uncensored_continued	I1-IQ2_XS	70.6B	19.69 GiB	1.41 GiB	22.22 GiB	0.10 GiB	12±22%
Llama-3.3-70B-Instruct-abliterated	I1-IQ2_XS	70.6B	19.69 GiB	1.41 GiB	22.22 GiB	0.10 GiB	12±22%
grok-oss-Revenant-70B	I1-IQ2_XS	70.6B	19.69 GiB	1.41 GiB	22.22 GiB	0.10 GiB	12±22%
Llama-3.1-Nemotron-70B-Instruct-HF	I1-IQ2_XS	70.6B	19.69 GiB	1.41 GiB	22.22 GiB	0.10 GiB	12±22%
L3.3-70B-Euryale-v2.3	I1-IQ2_XS	70.6B	19.69 GiB	1.41 GiB	22.22 GiB	0.10 GiB	12±22%
Hermes-3-Llama-3.1-70B	IQ2_XS	70.6B	19.69 GiB	1.41 GiB	22.22 GiB	0.10 GiB	12±22%
Hermes-4-70B-heretic	I1-IQ2_XS	70.6B	19.69 GiB	1.41 GiB	22.22 GiB	0.10 GiB	12±22%
Llama-3.3-70B-Instruct	IQ2_XS	70.6B	19.69 GiB	1.41 GiB	22.22 GiB	0.10 GiB	12±22%
Llama-3.1-70B	IQ2_XS	70.6B	19.69 GiB	1.41 GiB	22.22 GiB	0.10 GiB	12±22%
Anubis-70B-v1.2	IQ2_XS	70.6B	19.69 GiB	1.41 GiB	22.22 GiB	0.10 GiB	12±22%
Hermes-4-70B	IQ2_XS	70.6B	19.69 GiB	1.41 GiB	22.22 GiB	0.10 GiB	12±22%
Golem-70B-v1b	I1-IQ2_XS	70.6B	19.69 GiB	1.41 GiB	22.22 GiB	0.10 GiB	12±22%
DeepSeek-R1-Distill-Llama-70B-abliterated	I1-IQ2_XS	70.6B	19.69 GiB	1.41 GiB	22.22 GiB	0.10 GiB	12±22%
DeepSeek-R1-Distill-Llama-70B-heretic	I1-IQ2_XS	70.6B	19.69 GiB	1.41 GiB	22.22 GiB	0.10 GiB	12±22%
llama-3-firefunction-v2	IQ2_XS	70.6B	19.69 GiB	1.41 GiB	22.22 GiB	0.10 GiB	12±22%
DeepSeek-R1-Distill-Llama-70B	IQ2_XS	70.6B	19.69 GiB	1.41 GiB	22.22 GiB	0.10 GiB	12±22%
Legion-V2.1-LLaMa-70B	I1-IQ2_XS	70.6B	19.69 GiB	1.41 GiB	22.22 GiB	0.10 GiB	12±22%
Assistant_Pepe_70B	I1-IQ2_XS	70.6B	19.69 GiB	1.41 GiB	22.22 GiB	0.10 GiB	12±22%
Tess-R1-Limerick-Llama-3.1-70B	IQ2_XS	70.6B	19.69 GiB	1.41 GiB	22.22 GiB	0.10 GiB	12±22%

●From the file◐Predictedwhat these mean

Speed is modeled, not measured: decode is memory-bandwidth bound, so tokens per second is bytes read per token against achievable bandwidth. Mixture-of-experts models carry a wider band because only the routed experts are read each step, and few have been measured publicly.

Questions people ask

What AI models can a RTX 4500 Ada Generation run?: 1960 of 2118 indexed open-weight models fit a RTX 4500 Ada Generation at 16,384 context with q4_0 KV cache, the largest being Delphi-25B-SimpleRL-Math at I1-Q5_K_M. That covers text, vision-language, image, video and speech models.
How much usable memory does a RTX 4500 Ada Generation actually have?: Its nameplate is 24 GB, but about 22.32 GiB is available to a model once driver and compositor overhead is accounted for.
Is a RTX 4500 Ada Generation fast for local AI?: Its memory bandwidth is 432 GB/s, and that figure — not teraflops — is what governs token generation speed. Capacity decides what you can run; bandwidth decides how fast it runs.