NVIDIA · workstation

RTX A400

RTX A400 has 4 GB of VRAM at 96 GB/s — about 3.72 GiB usable after driver and compositor overhead. 794 of 2118 indexed models fit at 8K context with q4_0 KV.

Spec sheet· bandwidth, theoreticalFrom the file· fit from summed bytesPredicted· speed
Memory
4 GB
GDDR6
Bandwidth
96 GB/s
64-bit bus
Tensor FP16
11 TF
dense
TDP
50 W
$135 MSRP
Context4K8K16K32K64K128K
KV cachef16q8_0q4_0quantizing the KV cache is a ~2× lever on the dominant term at long context
vision language 54text 656embedding 25image 1audio asr 37audio tts 19video 2

What fits at 8K context

largest quantization that fits, per model · 794 of 2118 indexed
ModelBest quantParamsWeights●KV●Total in memory◐Headroom◐tok/s◐
gemma-3n-E4B-itUD-IQ2_XXS7.8B2.64 GiB0.05 GiB3.72 GiB0.00 GiB20±22%
LFM2.5-Audio-1.5B-JPF161.5B2.67 GiB0.00 GiB3.72 GiB0.00 GiB20±22%
INTELLECT-1-InstructI1-IQ1_S10.2B2.31 GiB0.37 GiB3.72 GiB0.00 GiB20±22%
NVIDIA-Nemotron-3-Nano-4B-BF16IQ3_M4.0B2.32 GiB0.37 GiB3.72 GiB0.00 GiB20±22%
Dolphin3.0-Llama3.2-3BQ6_K3.2B2.46 GiB0.25 GiB3.72 GiB0.00 GiB20±22%
Llama-Doctor-3.2-3B-InstructI1-Q6_K3.2B2.46 GiB0.25 GiB3.72 GiB0.00 GiB20±22%
Llama-Song-Stream-3B-InstructQ6_K3.2B2.46 GiB0.25 GiB3.72 GiB0.00 GiB20±22%
Llama-3.2-3B-Instruct-roleplay-tunedI1-Q6_K3.2B2.46 GiB0.25 GiB3.72 GiB0.00 GiB20±22%
Llama-3.2-3B-Instruct-heretic-ablitered-uncensoredI1-Q6_K3.2B2.46 GiB0.25 GiB3.72 GiB0.00 GiB20±22%
llama-3.2-Korean-Bllossom-3BQ6_K3.2B2.46 GiB0.25 GiB3.72 GiB0.00 GiB20±22%
llama-3.2-3b-instruct-bnb-4bitQ6_K3.3B2.46 GiB0.25 GiB3.72 GiB0.00 GiB20±22%
Llama-3.2-3B-InstructQ6_K3.2B2.46 GiB0.25 GiB3.72 GiB0.00 GiB20±22%
Llama-3.2-3BQ6_K3.2B2.46 GiB0.25 GiB3.72 GiB0.00 GiB20±22%
llama-3.2-3b-instructQ6_K3.2B2.46 GiB0.25 GiB3.72 GiB0.00 GiB20±22%
Llama3.2-3B-creative-writer-v0.1I1-Q6_K3.2B2.46 GiB0.25 GiB3.72 GiB0.00 GiB20±22%
Firefly-V3.2I1-Q6_K3.2B2.46 GiB0.25 GiB3.72 GiB0.00 GiB20±22%
Firefly-V3I1-Q6_K3.2B2.46 GiB0.25 GiB3.72 GiB0.00 GiB20±22%
Hermes-3-Llama-3.2-3BQ6_K3.2B2.46 GiB0.25 GiB3.72 GiB0.00 GiB20±22%
umt5-xxlQ3_K_S5.7B2.66 GiB0.00 GiB3.71 GiB0.01 GiB21±22%
Qwen2.5-Omni-7BUD-IQ2_M10.7B2.66 GiB0.00 GiB3.70 GiB0.02 GiB21±22%
MiMo-VL-7B-RLI1-IQ2_XS8.3B2.36 GiB0.32 GiB3.70 GiB0.02 GiB20±22%
Kuwutu-7B-CYOA-v2I1-IQ2_XS7.6B2.36 GiB0.32 GiB3.70 GiB0.02 GiB20±22%
Nemotron-3-Embed-8B-BF16Q2_08.0B2.36 GiB0.30 GiB3.70 GiB0.02 GiB21±22%
Yi-6B-ChatI1-IQ3_S6.1B2.53 GiB0.14 GiB3.70 GiB0.02 GiB20±22%
granite-4.0-7B-A1B-Creative-v0.1MoEI1-IQ3_S6.7B2.71 GiB0.02 GiB3.70 GiB0.02 GiB71±37%
Janus-Pro-7BI1-IQ1_S7.4B1.61 GiB1.05 GiB3.69 GiB0.03 GiB21±22%
deepseek-coder-7b-instruct-v1.5I1-IQ1_S6.9B1.61 GiB1.05 GiB3.69 GiB0.03 GiB21±22%
Qwen2.5-3BQ6_K3.1B2.60 GiB0.08 GiB3.69 GiB0.03 GiB20±22%
GRM-Kerlin-3bI1-Q6_K3.4B2.60 GiB0.08 GiB3.69 GiB0.03 GiB20±22%
Qwen2.5-Coder-3B-InstructQ6_K3.1B2.60 GiB0.08 GiB3.69 GiB0.03 GiB20±22%
Qwen2.5-3B-InstructQ6_K3.1B2.60 GiB0.08 GiB3.69 GiB0.03 GiB20±22%
LCO-Embedding-Omni-3B-2605Q6_K4.7B2.60 GiB0.08 GiB3.69 GiB0.03 GiB20±22%
Garnet-OCR-3B-0422I1-Q6_K4.1B2.60 GiB0.08 GiB3.69 GiB0.03 GiB20±22%
Ministral-3-8B-Instruct-2512-BF16-abliteratedI1-IQ2_XXS8.9B2.35 GiB0.30 GiB3.69 GiB0.03 GiB21±22%
Amaretto-8BI1-IQ2_XXS8.9B2.35 GiB0.30 GiB3.69 GiB0.03 GiB21±22%
alduin-4b-it-baseI1-Q5_K_S4.3B2.58 GiB0.09 GiB3.69 GiB0.03 GiB20±22%
Voxtral-Mini-3B-2507Q4_14.7B2.42 GiB0.26 GiB3.69 GiB0.03 GiB20±22%
moondream2F161.9B2.64 GiB0.00 GiB3.69 GiB0.03 GiB21±22%
Yi-1.5-6B-ChatQ3_K_S6.1B2.52 GiB0.14 GiB3.69 GiB0.03 GiB21±22%
deepseek-coder-6.7B-kexerI1-IQ1_M6.7B1.54 GiB1.13 GiB3.69 GiB0.03 GiB21±22%
Magicoder-S-DS-6.7BI1-IQ1_M6.7B1.54 GiB1.13 GiB3.69 GiB0.03 GiB21±22%
deepseek-coder-6.7b-baseI1-IQ1_M6.7B1.54 GiB1.13 GiB3.69 GiB0.03 GiB21±22%
WizardLM-7B-UncensoredI1-IQ1_M6.7B1.54 GiB1.13 GiB3.69 GiB0.03 GiB21±22%
Llama-2-7B-32K-InstructI1-IQ1_M6.7B1.54 GiB1.13 GiB3.69 GiB0.03 GiB21±22%
Luna-AI-Llama2-UncensoredI1-IQ1_M6.7B1.54 GiB1.13 GiB3.69 GiB0.03 GiB21±22%
Swallow-7b-NVE-instruct-hfI1-IQ1_M6.7B1.54 GiB1.13 GiB3.69 GiB0.03 GiB21±22%
Olmo-3-7B-InstructUD-IQ1_M7.3B1.90 GiB0.76 GiB3.68 GiB0.04 GiB21±22%
Olmo-3-7B-ThinkUD-IQ1_M7.3B1.90 GiB0.76 GiB3.68 GiB0.04 GiB21±22%
granite-3.3-2b-instructQ8_02.5B2.51 GiB0.18 GiB3.68 GiB0.04 GiB20±22%
granite-3.2-2b-instructQ8_02.5B2.51 GiB0.18 GiB3.68 GiB0.04 GiB20±22%
granite-vision-3.2-2bQ8_03.0B2.51 GiB0.18 GiB3.68 GiB0.04 GiB20±22%
gemma-3-4b-it-roleplay-tuned-v1I1-Q5_K_S4.3B2.57 GiB0.09 GiB3.68 GiB0.04 GiB21±22%
Gemma-3-4B-VL-it-Gemini-Pro-Heretic-Uncensored-ThinkingQ5_K_S4.3B2.57 GiB0.09 GiB3.68 GiB0.04 GiB21±22%
gemma-3-4b-it-roleplay-tuned-v2I1-Q5_K_S4.3B2.57 GiB0.09 GiB3.68 GiB0.04 GiB21±22%
medgemma-1.5-4b-itQ5_K_S4.3B2.57 GiB0.09 GiB3.68 GiB0.04 GiB21±22%
gemma-3-4b-it-heretic-uncensored-abliterated-ExtremeI1-Q5_K_S4.3B2.57 GiB0.09 GiB3.68 GiB0.04 GiB21±22%
medgemma-4b-itQ5_K_S4.3B2.57 GiB0.09 GiB3.68 GiB0.04 GiB21±22%
gemma-3-4b-it-abliteratedQ5_K_S4.3B2.57 GiB0.09 GiB3.68 GiB0.04 GiB21±22%
amoral-gemma3-4B-v1Q5_K_S4.3B2.57 GiB0.09 GiB3.68 GiB0.04 GiB21±22%
Gemma3-4B-CodeCenturionI1-Q5_K_S4.3B2.57 GiB0.09 GiB3.68 GiB0.04 GiB21±22%
●From the file◐Predictedwhat these mean

Speed is modeled, not measured: decode is memory-bandwidth bound, so tokens per second is bytes read per token against achievable bandwidth. Mixture-of-experts models carry a wider band because only the routed experts are read each step, and few have been measured publicly.

Questions people ask

What AI models can a RTX A400 run?
794 of 2118 indexed open-weight models fit a RTX A400 at 8,192 context with q4_0 KV cache, the largest being gemma-3n-E4B-it at UD-IQ2_XXS. That covers text, vision-language, image, video and speech models.
How much usable memory does a RTX A400 actually have?
Its nameplate is 4 GB, but about 3.72 GiB is available to a model once driver and compositor overhead is accounted for.
Is a RTX A400 fast for local AI?
Its memory bandwidth is 96 GB/s, and that figure — not teraflops — is what governs token generation speed. Capacity decides what you can run; bandwidth decides how fast it runs.