NVIDIA · workstation

RTX A400

RTX A400 has 4 GB of VRAM at 96 GB/s — about 3.72 GiB usable after driver and compositor overhead. 338 of 2118 indexed models fit at 128K context with q4_0 KV.

Spec sheet· bandwidth, theoreticalFrom the file· fit from summed bytesPredicted· speed
Memory
4 GB
GDDR6
Bandwidth
96 GB/s
64-bit bus
Tensor FP16
11 TF
dense
TDP
50 W
$135 MSRP
Context4K8K16K32K64K128K
KV cachef16q8_0q4_0quantizing the KV cache is a ~2× lever on the dominant term at long context
text 248vision language 29audio tts 17video 2embedding 13audio asr 29

What fits at 128K context

largest quantization that fits, per model · 338 of 2118 indexed
ModelBest quantParamsWeights●KV●Total in memory◐Headroom◐tok/s◐
starcoder2-3bKV unresolvedQ4_K_S3.0B1.64 GiB1.05 GiB3.72 GiB0.00 GiB20±22%
LFM2.5-Audio-1.5B-JPF161.5B2.67 GiB0.00 GiB3.72 GiB0.00 GiB20±22%
Vikhr-Gemma-2B-instructIQ1_M2.6B0.81 GiB1.89 GiB3.72 GiB0.00 GiB20±22%
Gemmasutra-Mini-2B-v1I1-IQ1_M2.6B0.81 GiB1.89 GiB3.72 GiB0.00 GiB20±22%
gemma-2-2b-itIQ1_M2.6B0.81 GiB1.89 GiB3.72 GiB0.00 GiB20±22%
umt5-xxlQ3_K_S5.7B2.66 GiB0.00 GiB3.71 GiB0.01 GiB21±22%
gemma-4-E2B-it-abliteratedI1-IQ3_XXS5.1B2.47 GiB0.25 GiB3.71 GiB0.01 GiB20±22%
gemma-4-E2B-it-qat-q4_0-unquantized-hereticI1-IQ3_XXS5.1B2.47 GiB0.25 GiB3.71 GiB0.01 GiB20±22%
Huihui-gemma-4-E2B-it-qat-q4_0-unquantized-abliteratedI1-IQ3_XXS5.1B2.47 GiB0.25 GiB3.71 GiB0.01 GiB20±22%
Gemma4_E2B_Abliterated_Baked_HF_ReadyI1-IQ3_XXS5.1B2.47 GiB0.25 GiB3.71 GiB0.01 GiB20±22%
gemma-4-E2B-itIQ3_XXS5.1B2.47 GiB0.25 GiB3.71 GiB0.01 GiB20±22%
granite-4.0-h-tinyMoEQ2_K6.9B2.46 GiB0.28 GiB3.71 GiB0.01 GiB48±37%
Qwen2.5-Omni-7BUD-IQ2_M10.7B2.66 GiB0.00 GiB3.70 GiB0.02 GiB21±22%
GLM-OCRIQ4_XS1.3B0.47 GiB2.25 GiB3.70 GiB0.02 GiB20±22%
LFM2-2.6B-ExpQ6_K_L2.6B2.13 GiB0.56 GiB3.70 GiB0.02 GiB20±22%
EXAONE-4.0-1.2B-abliteratedI1-IQ3_XXS1.5B0.61 GiB2.11 GiB3.70 GiB0.02 GiB20±22%
GRM-Kerlin-3bI1-IQ3_XS3.4B1.42 GiB1.27 GiB3.70 GiB0.02 GiB20±22%
Garnet-OCR-3B-0422I1-IQ3_XS4.1B1.42 GiB1.27 GiB3.70 GiB0.02 GiB20±22%
Darwin-4B-ChimeraI1-IQ3_XS4.0B1.69 GiB0.99 GiB3.69 GiB0.03 GiB20±22%
moondream2F161.9B2.64 GiB0.00 GiB3.69 GiB0.03 GiB21±22%
granite-3.1-1b-a400m-instructMoEQ6_K_L1.3B1.04 GiB1.69 GiB3.68 GiB0.04 GiB16±37%
VoxCPM2Q8_02.3B2.63 GiB0.00 GiB3.68 GiB0.04 GiB21±22%
Miril-Drone-2B-1IQ2_M5.1B2.43 GiB0.25 GiB3.68 GiB0.04 GiB20±22%
Qwen3.5-4B-NSFW-ARA-Heretic-LiteroticaI1-IQ2_S4.2B1.54 GiB1.13 GiB3.68 GiB0.04 GiB21±22%
Qwen3.5-4B-RpRMax-v1I1-IQ2_S4.7B1.54 GiB1.13 GiB3.68 GiB0.04 GiB21±22%
Holo-3.1-4B-uncensored-hereticI1-IQ2_S4.5B1.54 GiB1.13 GiB3.68 GiB0.04 GiB21±22%
GRaPE-2-MiniI1-IQ2_S4.7B1.54 GiB1.13 GiB3.68 GiB0.04 GiB21±22%
Qwen3.5-DPO-4B-2I1-IQ2_S4.2B1.54 GiB1.13 GiB3.68 GiB0.04 GiB21±22%
Huihui-Qwen3.5-4B-Claude-4.6-Opus-abliteratedI1-IQ2_S4.7B1.54 GiB1.13 GiB3.68 GiB0.04 GiB21±22%
Qwopus3.5-4B-v3-hereticI1-IQ2_S4.5B1.54 GiB1.13 GiB3.68 GiB0.04 GiB21±22%
Aureth-4B-Qwen3.5I1-IQ2_S4.5B1.54 GiB1.13 GiB3.68 GiB0.04 GiB21±22%
EXAONE-Deep-7.8BIQ2_M7.8B2.63 GiB0.00 GiB3.67 GiB0.05 GiB21±22%
EXAONE-3.5-7.8B-InstructIQ2_M7.8B2.63 GiB0.00 GiB3.67 GiB0.05 GiB21±22%
gemma-3n-E2B-itQ3_K_S5.4B2.23 GiB0.44 GiB3.67 GiB0.05 GiB21±22%
Qwen2-1.5BIQ4_XS1.5B1.68 GiB0.98 GiB3.66 GiB0.06 GiB21±22%
Dolphin3.0-Qwen2.5-3bIQ3_M3.1B1.39 GiB1.27 GiB3.66 GiB0.06 GiB21±22%
Qwen2.5-Coder-3B-Instruct-abliteratedI1-IQ3_M3.1B1.39 GiB1.27 GiB3.66 GiB0.06 GiB21±22%
GRM-Kerlin-3b-AbliteratedI1-IQ3_M3.1B1.39 GiB1.27 GiB3.66 GiB0.06 GiB21±22%
Qwen2.5-Coder-3B-InstructIQ3_M3.1B1.39 GiB1.27 GiB3.66 GiB0.06 GiB21±22%
Mythos-nanoI1-IQ3_M3.1B1.39 GiB1.27 GiB3.66 GiB0.06 GiB21±22%
MATE-3BI1-IQ3_M3.1B1.39 GiB1.27 GiB3.66 GiB0.06 GiB21±22%
Mythos-nano-OBLITERATEDI1-IQ3_M3.1B1.39 GiB1.27 GiB3.66 GiB0.06 GiB21±22%
Qwen2.5-3B-Instruct-UncensoredI1-IQ3_M3.1B1.39 GiB1.27 GiB3.66 GiB0.06 GiB21±22%
Qwen2.5-3B-InstructIQ3_M3.1B1.39 GiB1.27 GiB3.66 GiB0.06 GiB21±22%
Qwen2.5-3BIQ3_M3.1B1.39 GiB1.27 GiB3.66 GiB0.06 GiB21±22%
Qwen2.5-Coder-3BIQ3_M3.1B1.39 GiB1.27 GiB3.66 GiB0.06 GiB21±22%
raspberry-3BIQ3_M3.1B1.39 GiB1.27 GiB3.66 GiB0.06 GiB21±22%
VibeThinker-3B-OBLITERATEDI1-IQ3_M3.1B1.39 GiB1.27 GiB3.66 GiB0.06 GiB21±22%
Fourier-Qwen2.5-VL-3B-0.67I1-IQ3_M3.8B1.39 GiB1.27 GiB3.66 GiB0.06 GiB21±22%
Nanonets-OCR-sI1-IQ3_M3.8B1.39 GiB1.27 GiB3.66 GiB0.06 GiB21±22%
jina-embeddings-v4IQ3_M3.8B1.39 GiB1.27 GiB3.66 GiB0.06 GiB21±22%
granite-4.0-7B-A1B-Creative-v0.1MoEI1-IQ3_XXS6.7B2.42 GiB0.28 GiB3.66 GiB0.06 GiB48±37%
Holo-3.1-4BI1-IQ1_M5.2B1.52 GiB1.13 GiB3.66 GiB0.06 GiB21±22%
AfriqueQwen3.5-4BI1-IQ1_M5.2B1.52 GiB1.13 GiB3.66 GiB0.06 GiB21±22%
TimeOmni-1-4BI1-IQ1_M5.2B1.52 GiB1.13 GiB3.66 GiB0.06 GiB21±22%
granite-4.0-h-tiny-baseMoEQ2_K6.9B2.41 GiB0.28 GiB3.66 GiB0.06 GiB48±37%
Gemma-3-4b-it-Uncensored-DBL-XI1-Q2_K4.7B1.74 GiB0.90 GiB3.65 GiB0.07 GiB21±22%
Qwen2.5-Omni-3BQ6_K5.5B2.60 GiB0.00 GiB3.65 GiB0.07 GiB21±22%
Qwen3.5-4B-Claude-4.6-OS-Auto-Variable-HERETIC-UNCENSORED-THINKINGI1-IQ2_M4.5B1.51 GiB1.13 GiB3.65 GiB0.07 GiB21±22%
Qwen3.5-4B-SOMPOA-heresy-v2I1-IQ2_M4.5B1.51 GiB1.13 GiB3.65 GiB0.07 GiB21±22%
●From the file◐Predictedwhat these mean

Speed is modeled, not measured: decode is memory-bandwidth bound, so tokens per second is bytes read per token against achievable bandwidth. Mixture-of-experts models carry a wider band because only the routed experts are read each step, and few have been measured publicly.

Questions people ask

What AI models can a RTX A400 run?
338 of 2118 indexed open-weight models fit a RTX A400 at 131,072 context with q4_0 KV cache, the largest being starcoder2-3b at Q4_K_S. That covers text, vision-language, image, video and speech models.
How much usable memory does a RTX A400 actually have?
Its nameplate is 4 GB, but about 3.72 GiB is available to a model once driver and compositor overhead is accounted for.
Is a RTX A400 fast for local AI?
Its memory bandwidth is 96 GB/s, and that figure — not teraflops — is what governs token generation speed. Capacity decides what you can run; bandwidth decides how fast it runs.