NVIDIA · workstation
RTX A400
RTX A400 has 4 GB of VRAM at 96 GB/s — about 3.72 GiB usable after driver and compositor overhead. 339 of 2118 indexed models fit at 64K context with q8_0 KV.
Spec sheet· bandwidth, theoreticalFrom the file· fit from summed bytesPredicted· speed
Memory
4 GB
GDDR6
Bandwidth
96 GB/s
64-bit bus
Tensor FP16
11 TF
dense
TDP
50 W
$135 MSRP
text 249vision language 29audio tts 17video 2embedding 13audio asr 29
What fits at 64K context
largest quantization that fits, per model · 339 of 2118 indexed
| Model | Best quant | Params | Weights● | KV● | Total in memory◐ | Headroom◐ | tok/s◐ |
|---|---|---|---|---|---|---|---|
| GRM-Kerlin-3b | I1-IQ3_M | 3.4B | 1.51 GiB | 1.20 GiB | 3.72 GiB | 0.00 GiB | 20±22% |
| LFM2.5-Audio-1.5B-JP | F16 | 1.5B | 2.67 GiB | 0.00 GiB | 3.72 GiB | 0.00 GiB | 20±22% |
| Garnet-OCR-3B-0422 | I1-IQ3_M | 4.1B | 1.51 GiB | 1.20 GiB | 3.72 GiB | 0.00 GiB | 20±22% |
| umt5-xxl | Q3_K_S | 5.7B | 2.66 GiB | 0.00 GiB | 3.71 GiB | 0.01 GiB | 21±22% |
| granite-4.0-h-micro | Q6_K | 3.2B | 2.44 GiB | 0.27 GiB | 3.71 GiB | 0.01 GiB | 20±22% |
| gemma-4-E2B-it-abliterated | I1-IQ3_XXS | 5.1B | 2.47 GiB | 0.25 GiB | 3.71 GiB | 0.01 GiB | 20±22% |
| gemma-4-E2B-it-qat-q4_0-unquantized-heretic | I1-IQ3_XXS | 5.1B | 2.47 GiB | 0.25 GiB | 3.71 GiB | 0.01 GiB | 20±22% |
| Huihui-gemma-4-E2B-it-qat-q4_0-unquantized-abliterated | I1-IQ3_XXS | 5.1B | 2.47 GiB | 0.25 GiB | 3.71 GiB | 0.01 GiB | 20±22% |
| Gemma4_E2B_Abliterated_Baked_HF_Ready | I1-IQ3_XXS | 5.1B | 2.47 GiB | 0.25 GiB | 3.71 GiB | 0.01 GiB | 20±22% |
| gemma-4-E2B-it | IQ3_XXS | 5.1B | 2.47 GiB | 0.25 GiB | 3.71 GiB | 0.01 GiB | 20±22% |
| EXAONE-4.0-1.2B-abliterated | I1-Q3_K_M | 1.5B | 0.73 GiB | 1.99 GiB | 3.70 GiB | 0.02 GiB | 20±22% |
| Qwen2.5-Omni-7B | UD-IQ2_M | 10.7B | 2.66 GiB | 0.00 GiB | 3.70 GiB | 0.02 GiB | 21±22% |
| Holo-3.1-4B | I1-IQ2_XXS | 5.2B | 1.63 GiB | 1.06 GiB | 3.70 GiB | 0.02 GiB | 20±22% |
| AfriqueQwen3.5-4B | I1-IQ2_XXS | 5.2B | 1.63 GiB | 1.06 GiB | 3.70 GiB | 0.02 GiB | 20±22% |
| TimeOmni-1-4B | I1-IQ2_XXS | 5.2B | 1.63 GiB | 1.06 GiB | 3.70 GiB | 0.02 GiB | 20±22% |
| GLM-OCR | I1-Q5_K_M | 1.3B | 0.59 GiB | 2.13 GiB | 3.70 GiB | 0.02 GiB | 20±22% |
| Qwen3.5-4B-NSFW-ARA-Heretic-Literotica | I1-IQ2_M | 4.2B | 1.62 GiB | 1.06 GiB | 3.70 GiB | 0.02 GiB | 20±22% |
| Qwen3.5-4B-RpRMax-v1 | I1-IQ2_M | 4.7B | 1.62 GiB | 1.06 GiB | 3.70 GiB | 0.02 GiB | 20±22% |
| Holo-3.1-4B-uncensored-heretic | I1-IQ2_M | 4.5B | 1.62 GiB | 1.06 GiB | 3.70 GiB | 0.02 GiB | 20±22% |
| GRaPE-2-Mini | I1-IQ2_M | 4.7B | 1.62 GiB | 1.06 GiB | 3.70 GiB | 0.02 GiB | 20±22% |
| Qwen3.5-DPO-4B-2 | I1-IQ2_M | 4.2B | 1.62 GiB | 1.06 GiB | 3.70 GiB | 0.02 GiB | 20±22% |
| Huihui-Qwen3.5-4B-Claude-4.6-Opus-abliterated | I1-IQ2_M | 4.7B | 1.62 GiB | 1.06 GiB | 3.70 GiB | 0.02 GiB | 20±22% |
| Qwopus3.5-4B-v3-heretic | I1-IQ2_M | 4.5B | 1.62 GiB | 1.06 GiB | 3.70 GiB | 0.02 GiB | 20±22% |
| Aureth-4B-Qwen3.5 | I1-IQ2_M | 4.5B | 1.62 GiB | 1.06 GiB | 3.70 GiB | 0.02 GiB | 20±22% |
| DeepScaleR-1.5B-Preview | Q8_0 | 1.8B | 1.76 GiB | 0.93 GiB | 3.69 GiB | 0.03 GiB | 20±22% |
| Qwen2.5-Coder-1.5B-Instruct-abliterated | Q8_0 | 1.8B | 1.76 GiB | 0.93 GiB | 3.69 GiB | 0.03 GiB | 20±22% |
| Qwen2.5-1.5B-Instruct-uncensored | Q8_0 | 1.8B | 1.76 GiB | 0.93 GiB | 3.69 GiB | 0.03 GiB | 20±22% |
| VibeThinker-1.5B | Q8_0 | 1.8B | 1.76 GiB | 0.93 GiB | 3.69 GiB | 0.03 GiB | 20±22% |
| Nemotron-Research-Reasoning-Qwen-1.5B | Q8_0 | 1.8B | 1.76 GiB | 0.93 GiB | 3.69 GiB | 0.03 GiB | 20±22% |
| DeepSeek-R1-Distill-Qwen-1.5B-uncensored | Q8_0 | 1.8B | 1.76 GiB | 0.93 GiB | 3.69 GiB | 0.03 GiB | 20±22% |
| DeepSeek-R1-Distill-Qwen-1.5B | Q8_0 | 1.8B | 1.76 GiB | 0.93 GiB | 3.69 GiB | 0.03 GiB | 20±22% |
| Qwen2.5-Coder-1.5B-Instruct | Q8_0 | 1.5B | 1.76 GiB | 0.93 GiB | 3.69 GiB | 0.03 GiB | 20±22% |
| Qwen2.5-1.5B-Instruct | Q8_0 | 1.5B | 1.76 GiB | 0.93 GiB | 3.69 GiB | 0.03 GiB | 20±22% |
| deepseek-r1-distill-qwen-1.5b-unsloth-bnb-4bit | Q8_0 | 1.8B | 1.76 GiB | 0.93 GiB | 3.69 GiB | 0.03 GiB | 20±22% |
| dots.ocr | Q8_0 | 3.0B | 1.76 GiB | 0.93 GiB | 3.69 GiB | 0.03 GiB | 20±22% |
| DeepSeek-R1-Distill-Qwen-1.5B-Fully-Uncensored | Q8_0 | 1.8B | 1.76 GiB | 0.93 GiB | 3.69 GiB | 0.03 GiB | 20±22% |
| granite-4.0-h-tinyMoE | Q2_K | 6.9B | 2.46 GiB | 0.27 GiB | 3.69 GiB | 0.03 GiB | 49±37% |
| Qwen3.5-4B-Claude-4.6-OS-Auto-Variable-HERETIC-UNCENSORED-THINKING | I1-Q2_K_S | 4.5B | 1.62 GiB | 1.06 GiB | 3.69 GiB | 0.03 GiB | 20±22% |
| Qwen3.5-4B-SOMPOA-heresy-v2 | I1-Q2_K_S | 4.5B | 1.62 GiB | 1.06 GiB | 3.69 GiB | 0.03 GiB | 20±22% |
| Qwen3.5-4B-SOMPOA-heresy | I1-Q2_K_S | 4.5B | 1.62 GiB | 1.06 GiB | 3.69 GiB | 0.03 GiB | 20±22% |
| Qwen3.5-4B-Safety-Thinking | I1-Q2_K_S | 4.2B | 1.62 GiB | 1.06 GiB | 3.69 GiB | 0.03 GiB | 20±22% |
| Huihui-Qwen3.5-4B-abliterated | I1-Q2_K_S | 4.5B | 1.62 GiB | 1.06 GiB | 3.69 GiB | 0.03 GiB | 20±22% |
| Darkidol-Ballad-4B | I1-Q2_K_S | 4.5B | 1.62 GiB | 1.06 GiB | 3.69 GiB | 0.03 GiB | 20±22% |
| moondream2 | F16 | 1.9B | 2.64 GiB | 0.00 GiB | 3.69 GiB | 0.03 GiB | 21±22% |
| Dolphin3.0-Qwen2.5-3b | Q3_K_M | 3.1B | 1.48 GiB | 1.20 GiB | 3.69 GiB | 0.03 GiB | 20±22% |
| Qwen2.5-Coder-3B-Instruct-abliterated | I1-Q3_K_M | 3.1B | 1.48 GiB | 1.20 GiB | 3.69 GiB | 0.03 GiB | 20±22% |
| GRM-Kerlin-3b-Abliterated | I1-Q3_K_M | 3.1B | 1.48 GiB | 1.20 GiB | 3.69 GiB | 0.03 GiB | 20±22% |
| Mythos-nano | I1-Q3_K_M | 3.1B | 1.48 GiB | 1.20 GiB | 3.69 GiB | 0.03 GiB | 20±22% |
| MATE-3B | I1-Q3_K_M | 3.1B | 1.48 GiB | 1.20 GiB | 3.69 GiB | 0.03 GiB | 20±22% |
| Mythos-nano-OBLITERATED | I1-Q3_K_M | 3.1B | 1.48 GiB | 1.20 GiB | 3.69 GiB | 0.03 GiB | 20±22% |
| Qwen2.5-3B-Instruct-Uncensored | I1-Q3_K_M | 3.1B | 1.48 GiB | 1.20 GiB | 3.69 GiB | 0.03 GiB | 20±22% |
| Nanonets-OCR-s | Q3_K_M | 3.8B | 1.48 GiB | 1.20 GiB | 3.69 GiB | 0.03 GiB | 20±22% |
| Qwen2.5-Coder-3B | Q3_K_M | 3.1B | 1.48 GiB | 1.20 GiB | 3.69 GiB | 0.03 GiB | 20±22% |
| VibeThinker-3B-OBLITERATED | I1-Q3_K_M | 3.1B | 1.48 GiB | 1.20 GiB | 3.69 GiB | 0.03 GiB | 20±22% |
| raspberry-3B | Q3_K_M | 3.1B | 1.48 GiB | 1.20 GiB | 3.69 GiB | 0.03 GiB | 20±22% |
| VibeThinker-3B | Q3_K_M | 3.1B | 1.48 GiB | 1.20 GiB | 3.69 GiB | 0.03 GiB | 20±22% |
| Fourier-Qwen2.5-VL-3B-0.67 | I1-Q3_K_M | 3.8B | 1.48 GiB | 1.20 GiB | 3.69 GiB | 0.03 GiB | 20±22% |
| Qwen2.5-VL-3B-Instruct | Q3_K_M | 3.8B | 1.48 GiB | 1.20 GiB | 3.69 GiB | 0.03 GiB | 20±22% |
| jina-embeddings-v4 | Q3_K_M | 3.8B | 1.48 GiB | 1.20 GiB | 3.69 GiB | 0.03 GiB | 20±22% |
| Qwen2.5-3B | Q3_K_S | 3.1B | 1.48 GiB | 1.20 GiB | 3.69 GiB | 0.03 GiB | 20±22% |
Speed is modeled, not measured: decode is memory-bandwidth bound, so tokens per second is bytes read per token against achievable bandwidth. Mixture-of-experts models carry a wider band because only the routed experts are read each step, and few have been measured publicly.
Questions people ask
- What AI models can a RTX A400 run?
- 339 of 2118 indexed open-weight models fit a RTX A400 at 65,536 context with q8_0 KV cache, the largest being GRM-Kerlin-3b at I1-IQ3_M. That covers text, vision-language, image, video and speech models.
- How much usable memory does a RTX A400 actually have?
- Its nameplate is 4 GB, but about 3.72 GiB is available to a model once driver and compositor overhead is accounted for.
- Is a RTX A400 fast for local AI?
- Its memory bandwidth is 96 GB/s, and that figure — not teraflops — is what governs token generation speed. Capacity decides what you can run; bandwidth decides how fast it runs.