NVIDIA · workstation
RTX A2000
RTX A2000 has 6 GB of VRAM at 288 GB/s — about 5.58 GiB usable after driver and compositor overhead. 1222 of 2118 indexed models fit at 4K context with q4_0 KV.
Spec sheet· bandwidth, theoreticalFrom the file· fit from summed bytesPredicted· speed
Memory
6 GB
GDDR6
Bandwidth
288 GB/s
192-bit bus
Tensor FP16
32 TF
dense
TDP
70 W
$449 MSRP
text 1046vision language 90embedding 26audio asr 38image 1audio tts 19video 2
What fits at 4K context
largest quantization that fits, per model · 1222 of 2118 indexed
| Model | Best quant | Params | Weights● | KV● | Total in memory◐ | Headroom◐ | tok/s◐ |
|---|---|---|---|---|---|---|---|
| legitus-instruct-v1 | I1-IQ4_NL | 8.1B | 4.37 GiB | 0.14 GiB | 5.58 GiB | 0.00 GiB | 36±22% |
| Apertus-8B-Instruct-2509 | I1-IQ4_NL | 8.1B | 4.37 GiB | 0.14 GiB | 5.58 GiB | 0.00 GiB | 36±22% |
| Ministral-3-8B-Instruct-2512 | IQ4_XS | 8.9B | 4.39 GiB | 0.15 GiB | 5.58 GiB | 0.00 GiB | 36±22% |
| Ministral-3-8B-Reasoning-2512 | IQ4_XS | 8.9B | 4.39 GiB | 0.15 GiB | 5.58 GiB | 0.00 GiB | 36±22% |
| Mistral-7B-v0.3 | Q4_K_L | 7.2B | 4.40 GiB | 0.14 GiB | 5.58 GiB | 0.00 GiB | 36±22% |
| L3.2-Rogue-Creative-Instruct-Uncensored-Abliterated-7B | Q4_K_M | 7.5B | 4.27 GiB | 0.29 GiB | 5.58 GiB | 0.00 GiB | 36±22% |
| Qwen3.5-9B | Q3_K_M | 9.7B | 4.50 GiB | 0.04 GiB | 5.57 GiB | 0.01 GiB | 36±22% |
| Rocinante-XL-16B-v1 | I1-IQ2_XXS | 16.1B | 4.29 GiB | 0.24 GiB | 5.57 GiB | 0.01 GiB | 36±22% |
| granite-3.3-8b-instruct | Q4_K_S | 8.2B | 4.36 GiB | 0.18 GiB | 5.57 GiB | 0.01 GiB | 36±22% |
| granite-3.2-8b-instruct | Q4_K_S | 8.2B | 4.36 GiB | 0.18 GiB | 5.57 GiB | 0.01 GiB | 36±22% |
| MARTHA-LXVII.8B_QWEN-3.5-3.6_prune_9b-3.6_base | I1-Q4_0 | 8.1B | 4.50 GiB | 0.03 GiB | 5.57 GiB | 0.01 GiB | 36±22% |
| gemma-4-12B-it | UD-IQ3_XXS | 12.0B | 4.32 GiB | 0.20 GiB | 5.57 GiB | 0.01 GiB | 36±22% |
| gemma-4-12B-it-heretic | IQ3_XXS | 12.0B | 4.32 GiB | 0.20 GiB | 5.57 GiB | 0.01 GiB | 36±22% |
| Gemma-4-E4B-it-Minecraft-MT-en-zh-v0.1 | I1-Q3_K_M | 8.0B | 4.52 GiB | 0.03 GiB | 5.57 GiB | 0.01 GiB | 36±22% |
| gemma-4-E4B-Queen-it-qat-q4_0-unquantized | I1-Q3_K_M | 8.0B | 4.52 GiB | 0.03 GiB | 5.57 GiB | 0.01 GiB | 36±22% |
| Gemma-4-E4B-Luchador-Rudo | I1-Q3_K_M | 8.0B | 4.52 GiB | 0.03 GiB | 5.57 GiB | 0.01 GiB | 36±22% |
| supergemma4-e4b-abliterated | I1-Q3_K_M | 7.5B | 4.52 GiB | 0.03 GiB | 5.57 GiB | 0.01 GiB | 36±22% |
| Gemma-4-E4B-Abliterated | I1-Q3_K_M | 8.0B | 4.52 GiB | 0.03 GiB | 5.57 GiB | 0.01 GiB | 36±22% |
| gemma-4-E4B-it-The-DECKARD-Claude-Opus-Expresso-Universe-HERETIC-UNCENSORED-Thinking | I1-Q3_K_M | 8.0B | 4.52 GiB | 0.03 GiB | 5.57 GiB | 0.01 GiB | 36±22% |
| gemma-4-E4B-it-The-DECKARD-Expresso-Universe-HERETIC-UNCENSORED-Thinking | I1-Q3_K_M | 8.0B | 4.52 GiB | 0.03 GiB | 5.57 GiB | 0.01 GiB | 36±22% |
| gemma-4-E4B-it-heretic | I1-Q3_K_M | 8.0B | 4.52 GiB | 0.03 GiB | 5.57 GiB | 0.01 GiB | 36±22% |
| gemma-4-E4B-it-Claude-Opus-4.5-HERETIC-UNCENSORED-Thinking | I1-Q3_K_M | 8.0B | 4.52 GiB | 0.03 GiB | 5.57 GiB | 0.01 GiB | 36±22% |
| Huihui-gemma-4-E4B-it-abliterated | I1-Q3_K_M | 8.0B | 4.52 GiB | 0.03 GiB | 5.57 GiB | 0.01 GiB | 36±22% |
| gemma-4-E4B-it-Uncensored-MAX | I1-Q3_K_M | 8.0B | 4.52 GiB | 0.03 GiB | 5.57 GiB | 0.01 GiB | 36±22% |
| Darkidol-Gemma-4-E4B-it | I1-Q3_K_M | 8.0B | 4.52 GiB | 0.03 GiB | 5.57 GiB | 0.01 GiB | 36±22% |
| gemma-4-E4B-it-abliterated | I1-Q3_K_M | 8.0B | 4.52 GiB | 0.03 GiB | 5.57 GiB | 0.01 GiB | 36±22% |
| gemma-4-E4B-it-ultra-uncensored-heretic | Q3_K_M | 8.0B | 4.52 GiB | 0.03 GiB | 5.57 GiB | 0.01 GiB | 36±22% |
| gemma-4-E4B-it | Q3_K_M | 8.0B | 4.52 GiB | 0.03 GiB | 5.57 GiB | 0.01 GiB | 36±22% |
| OpenMedResearch-Gemma-4E4N | I1-Q3_K_M | 8.0B | 4.52 GiB | 0.03 GiB | 5.57 GiB | 0.01 GiB | 36±22% |
| Reasoning-Medical0.1-E4B-sft | I1-Q3_K_M | 8.0B | 4.52 GiB | 0.03 GiB | 5.57 GiB | 0.01 GiB | 36±22% |
| gemma-4-E4B | Q3_K_M | 8.0B | 4.52 GiB | 0.03 GiB | 5.57 GiB | 0.01 GiB | 36±22% |
| MiniCPM-Llama3-V-2_5 | IQ4_NL | 8.5B | 4.38 GiB | 0.14 GiB | 5.56 GiB | 0.02 GiB | 36±22% |
| Llama3-ChatQA-1.5-8B | IQ4_NL | 8.0B | 4.38 GiB | 0.14 GiB | 5.56 GiB | 0.02 GiB | 36±22% |
| grok-oss-Apollyon-8B | IQ4_NL | 8.0B | 4.38 GiB | 0.14 GiB | 5.56 GiB | 0.02 GiB | 36±22% |
| Turkish-Llama-8b-Instruct-v0.1 | IQ4_NL | 8.0B | 4.38 GiB | 0.14 GiB | 5.56 GiB | 0.02 GiB | 36±22% |
| Crow-9B-HERETIC-4.6 | I1-Q3_K_L | 9.4B | 4.49 GiB | 0.04 GiB | 5.56 GiB | 0.02 GiB | 36±22% |
| Qwen3.5-9B-Claude-4.6-OS-Auto-Variable-HERETIC-UNCENSORED-THINKING | I1-Q3_K_L | 9.4B | 4.49 GiB | 0.04 GiB | 5.56 GiB | 0.02 GiB | 36±22% |
| Qwen3.5-9B-Claude-4.6-HighIQ-INSTRUCT-HERETIC-UNCENSORED | I1-Q3_K_L | 9.4B | 4.49 GiB | 0.04 GiB | 5.56 GiB | 0.02 GiB | 36±22% |
| Qwen3.5-9B-Claude-4.6-OS-HERETIC-UNCENSORED-INSTRUCT | I1-Q3_K_L | 9.4B | 4.49 GiB | 0.04 GiB | 5.56 GiB | 0.02 GiB | 36±22% |
| Qwen3.5-9B-Claude-4.6-HighIQ-THINKING-HERETIC-UNCENSORED | I1-Q3_K_L | 9.4B | 4.49 GiB | 0.04 GiB | 5.56 GiB | 0.02 GiB | 36±22% |
| NaNovel-9B | I1-Q3_K_L | 9.7B | 4.49 GiB | 0.04 GiB | 5.56 GiB | 0.02 GiB | 36±22% |
| Qwen3.5-9B-Unredacted-MAX | I1-Q3_K_L | 9.4B | 4.49 GiB | 0.04 GiB | 5.56 GiB | 0.02 GiB | 36±22% |
| Qwen3.5-9B-abliterated | I1-Q3_K_L | 9.4B | 4.49 GiB | 0.04 GiB | 5.56 GiB | 0.02 GiB | 36±22% |
| Ken3.5-9B | I1-Q3_K_L | 9.7B | 4.49 GiB | 0.04 GiB | 5.56 GiB | 0.02 GiB | 36±22% |
| Huihui-Qwen3.5-9B-abliterated | Q3_K_L | 9.7B | 4.49 GiB | 0.04 GiB | 5.56 GiB | 0.02 GiB | 36±22% |
| Qwen3.5-9B-abliterated | Q3_K_L | 9.0B | 4.49 GiB | 0.04 GiB | 5.56 GiB | 0.02 GiB | 36±22% |
| Qwen3.5-9B-Base | Q3_K_L | 9.7B | 4.49 GiB | 0.04 GiB | 5.56 GiB | 0.02 GiB | 36±22% |
| Qwen3.5-9B-gemini-3.1-opus-4.6-reasoning | I1-Q3_K_L | 9.4B | 4.49 GiB | 0.04 GiB | 5.56 GiB | 0.02 GiB | 36±22% |
| LFM2-8B-A1BMoE | Q4_K_S | 8.3B | 4.56 GiB | 0.01 GiB | 5.56 GiB | 0.02 GiB | 108±37% |
| OLMoE-1B-7B-0924-InstructMoE | I1-Q5_K_S | 6.9B | 4.45 GiB | 0.14 GiB | 5.56 GiB | 0.02 GiB | 96±37% |
| Ministral-3-8B-Instruct-2512-BF16-abliterated | I1-IQ4_XS | 8.9B | 4.37 GiB | 0.15 GiB | 5.56 GiB | 0.02 GiB | 36±22% |
| Ministral-3-8B-Instruct-2512-BF16 | IQ4_XS | 8.9B | 4.37 GiB | 0.15 GiB | 5.56 GiB | 0.02 GiB | 36±22% |
| Amaretto-8B | I1-IQ4_XS | 8.9B | 4.37 GiB | 0.15 GiB | 5.56 GiB | 0.02 GiB | 36±22% |
| gemma-4-E4B-it | Q4_K_S | 8.0B | 4.51 GiB | 0.03 GiB | 5.56 GiB | 0.02 GiB | 36±22% |
| GLM-4.7-Flash-REAP-23B-A3B-absolute-heresyMoE | I1-IQ1_S | 23.0B | 4.49 GiB | 0.06 GiB | 5.56 GiB | 0.02 GiB | 129±37% |
| granite-3.1-8b-instruct | IQ4_NL | 8.2B | 4.35 GiB | 0.18 GiB | 5.56 GiB | 0.02 GiB | 36±22% |
| Kuwutu-7B-CYOA-v2 | Q4_K_M | 7.6B | 4.37 GiB | 0.16 GiB | 5.56 GiB | 0.02 GiB | 36±22% |
| Assistant_Pepe_8B | Q4_K_S | — | 4.37 GiB | 0.14 GiB | 5.55 GiB | 0.03 GiB | 36±22% |
| Ministral-8B-Instruct-2410 | Q4_K_S | 8.0B | 4.36 GiB | 0.16 GiB | 5.55 GiB | 0.03 GiB | 36±22% |
| Foundation-Sec-8B-Instruct | I1-Q4_K_S | 8.0B | 4.37 GiB | 0.14 GiB | 5.55 GiB | 0.03 GiB | 36±22% |
Speed is modeled, not measured: decode is memory-bandwidth bound, so tokens per second is bytes read per token against achievable bandwidth. Mixture-of-experts models carry a wider band because only the routed experts are read each step, and few have been measured publicly.
Questions people ask
- What AI models can a RTX A2000 run?
- 1222 of 2118 indexed open-weight models fit a RTX A2000 at 4,096 context with q4_0 KV cache, the largest being legitus-instruct-v1 at I1-IQ4_NL. That covers text, vision-language, image, video and speech models.
- How much usable memory does a RTX A2000 actually have?
- Its nameplate is 6 GB, but about 5.58 GiB is available to a model once driver and compositor overhead is accounted for.
- Is a RTX A2000 fast for local AI?
- Its memory bandwidth is 288 GB/s, and that figure — not teraflops — is what governs token generation speed. Capacity decides what you can run; bandwidth decides how fast it runs.