NVIDIA · workstation

RTX A2000

RTX A2000 has 6 GB of VRAM at 288 GB/s — about 5.58 GiB usable after driver and compositor overhead. 557 of 2118 indexed models fit at 128K context with q4_0 KV.

Spec sheet· bandwidth, theoreticalFrom the file· fit from summed bytesPredicted· speed
Memory
6 GB
GDDR6
Bandwidth
288 GB/s
192-bit bus
Tensor FP16
32 TF
dense
TDP
70 W
$449 MSRP
Context4K8K16K32K64K128K
KV cachef16q8_0q4_0quantizing the KV cache is a ~2× lever on the dominant term at long context
text 429vision language 52audio asr 36embedding 21audio tts 17video 2

What fits at 128K context

largest quantization that fits, per model · 557 of 2118 indexed
ModelBest quantParamsWeights●KV●Total in memory◐Headroom◐tok/s◐
Supertron2-Reranker-2BI1-IQ2_M2.1B0.65 GiB3.94 GiB5.58 GiB0.00 GiB35±22%
Uni-MuMER-Qwen3-VL-2BI1-IQ2_M2.1B0.65 GiB3.94 GiB5.58 GiB0.00 GiB35±22%
Qwen3-VL-Reranker-2BI1-IQ2_M2.1B0.65 GiB3.94 GiB5.58 GiB0.00 GiB35±22%
OpenCaption-2B-VL-SFT-v1.0I1-IQ2_M2.1B0.65 GiB3.94 GiB5.58 GiB0.00 GiB35±22%
Atomight-V2.5-1.7BI1-IQ2_M1.7B0.65 GiB3.94 GiB5.58 GiB0.00 GiB35±22%
OpenClaude-1.7B-MergedI1-IQ2_M1.7B0.65 GiB3.94 GiB5.58 GiB0.00 GiB35±22%
Qwen3-VL-2B-InstructIQ2_M2.1B0.65 GiB3.94 GiB5.58 GiB0.00 GiB35±22%
gaon-1.7b-v2-translateI1-IQ2_M1.7B0.65 GiB3.94 GiB5.58 GiB0.00 GiB35±22%
gaon-1.7b-v2-instructI1-IQ2_M1.7B0.65 GiB3.94 GiB5.58 GiB0.00 GiB35±22%
Lightning-1.7BIQ2_M1.7B0.65 GiB3.94 GiB5.58 GiB0.00 GiB35±22%
DorsetHeatwaveLLM2I1-IQ2_M1.7B0.65 GiB3.94 GiB5.58 GiB0.00 GiB35±22%
Ministral-3-3B-Instruct-2512UD-IQ1_S3.8B0.90 GiB3.66 GiB5.57 GiB0.01 GiB36±22%
Ministral-3-3B-Reasoning-2512UD-IQ1_S4.3B0.90 GiB3.66 GiB5.57 GiB0.01 GiB36±22%
Fara1.5-4BQ6_K4.5B3.42 GiB1.13 GiB5.56 GiB0.02 GiB36±22%
AREX-TurboQ6_K4.5B3.42 GiB1.13 GiB5.56 GiB0.02 GiB36±22%
Qwen3-0.6BQ6_K_L752M0.65 GiB3.94 GiB5.56 GiB0.02 GiB35±22%
ARK-ASR-3BQ4_K4.1B3.28 GiB1.27 GiB5.56 GiB0.02 GiB36±22%
Parable-Granite-4.1-3B-Claude-Fable-5I1-Q3_K_L3.4B1.74 GiB2.81 GiB5.55 GiB0.03 GiB36±22%
granite-4.0-microQ3_K_L3.4B1.74 GiB2.81 GiB5.55 GiB0.03 GiB36±22%
granite-4.1-3bQ3_K_L3.4B1.74 GiB2.81 GiB5.55 GiB0.03 GiB36±22%
granite-4.0-micro-baseQ3_K_L3.4B1.74 GiB2.81 GiB5.55 GiB0.03 GiB36±22%
SmolLM3-3BQ5_K_S3.1B2.01 GiB2.53 GiB5.55 GiB0.03 GiB36±22%
Tini-Cybersec-8B-A1BMoEQ3_K_L8.5B4.13 GiB0.42 GiB5.55 GiB0.03 GiB78±37%
LFM2.5-Queen-Opus-4.7-8B-A1BMoEI1-Q3_K_L8.5B4.13 GiB0.42 GiB5.55 GiB0.03 GiB78±37%
LFM2.5-8B-A1B-KO-SFTMoEI1-Q3_K_L8.5B4.13 GiB0.42 GiB5.55 GiB0.03 GiB78±37%
LFM2.5-8B-A1B-hereticMoEI1-Q3_K_L8.5B4.13 GiB0.42 GiB5.55 GiB0.03 GiB78±37%
LFM2.5-8B-A1B-SOMPOA-heresyMoEI1-Q3_K_L8.5B4.13 GiB0.42 GiB5.55 GiB0.03 GiB78±37%
Huihui-LFM2.5-8B-A1B-abliteratedMoEI1-Q3_K_L8.5B4.13 GiB0.42 GiB5.55 GiB0.03 GiB78±37%
Supertron2.1-8B-A1BMoEI1-Q3_K_L8.5B4.13 GiB0.42 GiB5.55 GiB0.03 GiB78±37%
Crow-9B-HERETIC-4.6I1-Q2_K9.4B3.39 GiB1.13 GiB5.55 GiB0.03 GiB36±22%
Qwen3.5-9B-Claude-4.6-OS-Auto-Variable-HERETIC-UNCENSORED-THINKINGI1-Q2_K9.4B3.39 GiB1.13 GiB5.55 GiB0.03 GiB36±22%
Qwen3.5-9B-Claude-4.6-HighIQ-INSTRUCT-HERETIC-UNCENSOREDI1-Q2_K9.4B3.39 GiB1.13 GiB5.55 GiB0.03 GiB36±22%
Qwen3.5-9B-Claude-4.6-OS-HERETIC-UNCENSORED-INSTRUCTI1-Q2_K9.4B3.39 GiB1.13 GiB5.55 GiB0.03 GiB36±22%
Qwen3.5-9B-Claude-4.6-HighIQ-THINKING-HERETIC-UNCENSOREDI1-Q2_K9.4B3.39 GiB1.13 GiB5.55 GiB0.03 GiB36±22%
NaNovel-9BI1-Q2_K9.7B3.39 GiB1.13 GiB5.55 GiB0.03 GiB36±22%
Qwen3.5-9B-Unredacted-MAXI1-Q2_K9.4B3.39 GiB1.13 GiB5.55 GiB0.03 GiB36±22%
Qwen3.5-9B-abliteratedI1-Q2_K9.4B3.39 GiB1.13 GiB5.55 GiB0.03 GiB36±22%
Ken3.5-9BI1-Q2_K9.7B3.39 GiB1.13 GiB5.55 GiB0.03 GiB36±22%
Huihui-Qwen3.5-9B-abliteratedQ2_K9.7B3.39 GiB1.13 GiB5.55 GiB0.03 GiB36±22%
Qwen3.5-9B-BaseQ2_K9.7B3.39 GiB1.13 GiB5.55 GiB0.03 GiB36±22%
Qwen3.5-9B-gemini-3.1-opus-4.6-reasoningI1-Q2_K9.4B3.39 GiB1.13 GiB5.55 GiB0.03 GiB36±22%
granite-4.0-7B-A1B-Creative-v0.1MoEI1-Q5_K_S6.7B4.30 GiB0.28 GiB5.54 GiB0.04 GiB96±37%
Felldude-Uncensored-Ministral3-3B-bf16I1-IQ1_S3.8B0.87 GiB3.66 GiB5.54 GiB0.04 GiB36±22%
Amaretto-3BI1-IQ1_S4.3B0.87 GiB3.66 GiB5.54 GiB0.04 GiB36±22%
Nanbeige4.1-3BQ4_K_M3.9B2.28 GiB2.25 GiB5.54 GiB0.04 GiB36±22%
GLM-4.6V-FlashUD-IQ1_M10.3B3.09 GiB1.41 GiB5.53 GiB0.05 GiB36±22%
GLM-Z1-9B-0414UD-IQ1_M9.4B3.09 GiB1.41 GiB5.53 GiB0.05 GiB36±22%
GLM-4-9B-0414UD-IQ1_M9.4B3.09 GiB1.41 GiB5.53 GiB0.05 GiB36±22%
GLM-4.1V-9B-ThinkingUD-IQ1_M10.3B3.09 GiB1.41 GiB5.53 GiB0.05 GiB36±22%
Unlimited-OCRMoEKV unresolvedQ6_K3.3B2.43 GiB2.11 GiB5.53 GiB0.05 GiB38±37%
granite-speech-4.1-2b-plusQ6_K2.1B1.73 GiB2.81 GiB5.53 GiB0.05 GiB36±22%
Yi-6B-ChatI1-IQ3_XXS6.1B2.25 GiB2.25 GiB5.52 GiB0.06 GiB36±22%
gemma-3n-E4B-itIQ4_XS7.8B3.98 GiB0.51 GiB5.52 GiB0.06 GiB36±22%
Vero-Qwen35-9B-BaseI1-IQ2_M9.4B3.36 GiB1.13 GiB5.52 GiB0.06 GiB36±22%
Vero-Qwen35-9BI1-IQ2_M9.4B3.36 GiB1.13 GiB5.52 GiB0.06 GiB36±22%
Qwen3.5-9B-Claude-4.6-Opus-Deckard-V4.2-Uncensored-Heretic-ThinkingI1-IQ2_M9.4B3.36 GiB1.13 GiB5.52 GiB0.06 GiB36±22%
Morphos-9BI1-IQ2_M9.0B3.36 GiB1.13 GiB5.52 GiB0.06 GiB36±22%
Qwable-9B-Claude-Fable-5-hereticI1-IQ2_M9.4B3.36 GiB1.13 GiB5.52 GiB0.06 GiB36±22%
Holo-3.1-9BI1-IQ2_M9.4B3.36 GiB1.13 GiB5.52 GiB0.06 GiB36±22%
Qwable-9B-Claude-Fable-5I1-IQ2_M9.4B3.36 GiB1.13 GiB5.52 GiB0.06 GiB36±22%
●From the file◐Predictedwhat these mean

Speed is modeled, not measured: decode is memory-bandwidth bound, so tokens per second is bytes read per token against achievable bandwidth. Mixture-of-experts models carry a wider band because only the routed experts are read each step, and few have been measured publicly.

Questions people ask

What AI models can a RTX A2000 run?
557 of 2118 indexed open-weight models fit a RTX A2000 at 131,072 context with q4_0 KV cache, the largest being Supertron2-Reranker-2B at I1-IQ2_M. That covers text, vision-language, image, video and speech models.
How much usable memory does a RTX A2000 actually have?
Its nameplate is 6 GB, but about 5.58 GiB is available to a model once driver and compositor overhead is accounted for.
Is a RTX A2000 fast for local AI?
Its memory bandwidth is 288 GB/s, and that figure — not teraflops — is what governs token generation speed. Capacity decides what you can run; bandwidth decides how fast it runs.