NVIDIA · workstation

RTX A400

RTX A400 has 4 GB of VRAM at 96 GB/s — about 3.72 GiB usable after driver and compositor overhead. 430 of 2118 indexed models fit at 32K context with q8_0 KV.

Spec sheet· bandwidth, theoreticalFrom the file· fit from summed bytesPredicted· speed
Memory
4 GB
GDDR6
Bandwidth
96 GB/s
64-bit bus
Tensor FP16
11 TF
dense
TDP
50 W
$135 MSRP
Context4K8K16K32K64K128K
KV cachef16q8_0q4_0quantizing the KV cache is a ~2× lever on the dominant term at long context
text 317vision language 42audio asr 32audio tts 17embedding 20video 2

What fits at 32K context

largest quantization that fits, per model · 430 of 2118 indexed
ModelBest quantParamsWeights●KV●Total in memory◐Headroom◐tok/s◐
Vikhr-Gemma-2B-instructQ4_K_L2.6B1.72 GiB0.98 GiB3.72 GiB0.00 GiB20±22%
gemma-2-2b-it-abliteratedQ4_K_L2.6B1.72 GiB0.98 GiB3.72 GiB0.00 GiB20±22%
Gemmasutra-Mini-2B-v1Q4_K_L2.6B1.72 GiB0.98 GiB3.72 GiB0.00 GiB20±22%
LFM2.5-Audio-1.5B-JPF161.5B2.67 GiB0.00 GiB3.72 GiB0.00 GiB20±22%
Llama-3.2-3B-InstructUD-IQ1_S3.2B0.85 GiB1.86 GiB3.72 GiB0.00 GiB20±22%
OpenClaude-1.7B-MergedIQ3_M1.7B0.86 GiB1.86 GiB3.72 GiB0.00 GiB20±22%
umt5-xxlQ3_K_S5.7B2.66 GiB0.00 GiB3.71 GiB0.01 GiB21±22%
LFM2-1.2BBF161.2B2.18 GiB0.53 GiB3.71 GiB0.01 GiB20±22%
Qwen2.5-3B-Instruct-abliteratedI1-IQ2_S3.1B2.10 GiB0.60 GiB3.71 GiB0.01 GiB20±22%
LFM2.5-8B-A1BMoEUD-IQ2_XXS8.5B2.52 GiB0.20 GiB3.71 GiB0.01 GiB48±37%
Fara1.5-4BQ3_K_S4.5B2.17 GiB0.53 GiB3.71 GiB0.01 GiB20±22%
AREX-TurboQ3_K_S4.5B2.17 GiB0.53 GiB3.71 GiB0.01 GiB20±22%
GLM-OCRF161.3B1.66 GiB1.06 GiB3.71 GiB0.01 GiB20±22%
Qwen2.5-Omni-7BUD-IQ2_M10.7B2.66 GiB0.00 GiB3.70 GiB0.02 GiB21±22%
FrickFritz-4BI1-Q3_K_M4.7B2.16 GiB0.53 GiB3.70 GiB0.02 GiB20±22%
qwen3.5-4b-agentic-coder-v4I1-Q3_K_M4.7B2.16 GiB0.53 GiB3.70 GiB0.02 GiB20±22%
Newton-bot-3-VLM-mini-4BQ3_K_M4.7B2.16 GiB0.53 GiB3.70 GiB0.02 GiB20±22%
Myth-4BI1-Q3_K_M4.3B2.16 GiB0.53 GiB3.70 GiB0.02 GiB20±22%
Qwen3.5-4B-UncensoredI1-Q3_K_M4.7B2.16 GiB0.53 GiB3.70 GiB0.02 GiB20±22%
JOSIE-2-4B-PreviewI1-Q3_K_M4.7B2.16 GiB0.53 GiB3.70 GiB0.02 GiB20±22%
Surogate-3.5-4BI1-Q3_K_M5.3B2.16 GiB0.53 GiB3.70 GiB0.02 GiB20±22%
Qwopus3.5-4B-v3Q3_K_M4.7B2.16 GiB0.53 GiB3.70 GiB0.02 GiB20±22%
granite-3.1-2b-instructQ4_K_S2.5B1.38 GiB1.33 GiB3.70 GiB0.02 GiB20±22%
granite-speech-4.1-2b-plusQ4_K_M2.1B1.39 GiB1.33 GiB3.70 GiB0.02 GiB20±22%
Yi-6B-ChatI1-IQ2_XXS6.1B1.61 GiB1.06 GiB3.70 GiB0.02 GiB20±22%
Darwin-4B-ChimeraQ3_K_M4.0B2.01 GiB0.68 GiB3.70 GiB0.02 GiB20±22%
granite-3.3-2b-instructQ4_K_S2.5B1.36 GiB1.33 GiB3.69 GiB0.03 GiB20±22%
granite-3.2-2b-instructQ4_K_S2.5B1.36 GiB1.33 GiB3.69 GiB0.03 GiB20±22%
granite-vision-3.2-2bQ4_K_S3.0B1.36 GiB1.33 GiB3.69 GiB0.03 GiB20±22%
moondream2F161.9B2.64 GiB0.00 GiB3.69 GiB0.03 GiB21±22%
granite-3.1-3b-a800m-instructMoEIQ4_XS3.3B1.66 GiB1.06 GiB3.69 GiB0.03 GiB21±37%
Ministral-3-3B-Instruct-2512UD-IQ1_M3.8B0.95 GiB1.73 GiB3.69 GiB0.03 GiB20±22%
Ministral-3-3B-Reasoning-2512UD-IQ1_M4.3B0.95 GiB1.73 GiB3.69 GiB0.03 GiB20±22%
Supertron2-Reranker-2BI1-IQ3_M2.1B0.83 GiB1.86 GiB3.69 GiB0.03 GiB20±22%
Uni-MuMER-Qwen3-VL-2BI1-IQ3_M2.1B0.83 GiB1.86 GiB3.69 GiB0.03 GiB20±22%
Qwen3-VL-Reranker-2BI1-IQ3_M2.1B0.83 GiB1.86 GiB3.69 GiB0.03 GiB20±22%
OpenCaption-2B-VL-SFT-v1.0I1-IQ3_M2.1B0.83 GiB1.86 GiB3.69 GiB0.03 GiB20±22%
Atomight-V2.5-1.7BI1-IQ3_M1.7B0.83 GiB1.86 GiB3.69 GiB0.03 GiB20±22%
Qwen3-VL-2B-InstructIQ3_M2.1B0.83 GiB1.86 GiB3.69 GiB0.03 GiB20±22%
gaon-1.7b-v2-translateI1-IQ3_M1.7B0.83 GiB1.86 GiB3.69 GiB0.03 GiB20±22%
gaon-1.7b-v2-instructI1-IQ3_M1.7B0.83 GiB1.86 GiB3.69 GiB0.03 GiB20±22%
Lightning-1.7BIQ3_M1.7B0.83 GiB1.86 GiB3.69 GiB0.03 GiB20±22%
DorsetHeatwaveLLM2I1-IQ3_M1.7B0.83 GiB1.86 GiB3.69 GiB0.03 GiB20±22%
Dolphin3.0-Qwen2.5-3bQ5_K_M3.1B2.07 GiB0.60 GiB3.68 GiB0.04 GiB21±22%
Qwen2.5-Coder-3B-Instruct-abliteratedI1-Q5_K_M3.1B2.07 GiB0.60 GiB3.68 GiB0.04 GiB21±22%
GRM-Kerlin-3b-AbliteratedI1-Q5_K_M3.1B2.07 GiB0.60 GiB3.68 GiB0.04 GiB21±22%
Mythos-nanoI1-Q5_K_M3.1B2.07 GiB0.60 GiB3.68 GiB0.04 GiB21±22%
MATE-3BI1-Q5_K_M3.1B2.07 GiB0.60 GiB3.68 GiB0.04 GiB21±22%
Mythos-nano-OBLITERATEDI1-Q5_K_M3.1B2.07 GiB0.60 GiB3.68 GiB0.04 GiB21±22%
Qwen2.5-3B-Instruct-UncensoredI1-Q5_K_M3.1B2.07 GiB0.60 GiB3.68 GiB0.04 GiB21±22%
Nanonets-OCR-sQ5_K_M3.8B2.07 GiB0.60 GiB3.68 GiB0.04 GiB21±22%
Qwen2.5-Coder-3BQ5_K_M3.1B2.07 GiB0.60 GiB3.68 GiB0.04 GiB21±22%
raspberry-3BQ5_K_M3.1B2.07 GiB0.60 GiB3.68 GiB0.04 GiB21±22%
VibeThinker-3B-OBLITERATEDI1-Q5_K_M3.1B2.07 GiB0.60 GiB3.68 GiB0.04 GiB21±22%
VibeThinker-3BQ5_K_M3.1B2.07 GiB0.60 GiB3.68 GiB0.04 GiB21±22%
Fourier-Qwen2.5-VL-3B-0.67I1-Q5_K_M3.8B2.07 GiB0.60 GiB3.68 GiB0.04 GiB21±22%
Qwen2.5-VL-3B-InstructQ5_K_M3.8B2.07 GiB0.60 GiB3.68 GiB0.04 GiB21±22%
jina-embeddings-v4Q5_K_M3.8B2.07 GiB0.60 GiB3.68 GiB0.04 GiB21±22%
Qwen3-1.7BIQ3_XXS2.0B0.83 GiB1.86 GiB3.68 GiB0.04 GiB20±22%
granite-4.0-7B-A1B-Creative-v0.1MoEI1-IQ3_XS6.7B2.58 GiB0.13 GiB3.68 GiB0.04 GiB58±37%
●From the file◐Predictedwhat these mean

Speed is modeled, not measured: decode is memory-bandwidth bound, so tokens per second is bytes read per token against achievable bandwidth. Mixture-of-experts models carry a wider band because only the routed experts are read each step, and few have been measured publicly.

Questions people ask

What AI models can a RTX A400 run?
430 of 2118 indexed open-weight models fit a RTX A400 at 32,768 context with q8_0 KV cache, the largest being Vikhr-Gemma-2B-instruct at Q4_K_L. That covers text, vision-language, image, video and speech models.
How much usable memory does a RTX A400 actually have?
Its nameplate is 4 GB, but about 3.72 GiB is available to a model once driver and compositor overhead is accounted for.
Is a RTX A400 fast for local AI?
Its memory bandwidth is 96 GB/s, and that figure — not teraflops — is what governs token generation speed. Capacity decides what you can run; bandwidth decides how fast it runs.