NVIDIA · workstation

RTX A2000

RTX A2000 has 6 GB of VRAM at 288 GB/s — about 5.58 GiB usable after driver and compositor overhead. 1085 of 2118 indexed models fit at 32K context with q4_0 KV.

Spec sheet· bandwidth, theoreticalFrom the file· fit from summed bytesPredicted· speed
Memory
6 GB
GDDR6
Bandwidth
288 GB/s
192-bit bus
Tensor FP16
32 TF
dense
TDP
70 W
$449 MSRP
Context4K8K16K32K64K128K
KV cachef16q8_0q4_0quantizing the KV cache is a ~2× lever on the dominant term at long context
text 916embedding 26vision language 84audio asr 38audio tts 19video 2

What fits at 32K context

largest quantization that fits, per model · 1085 of 2118 indexed
ModelBest quantParamsWeights●KV●Total in memory◐Headroom◐tok/s◐
Foundation-Sec-8B-InstructI1-Q3_K_S8.0B3.41 GiB1.13 GiB5.58 GiB0.00 GiB36±22%
Foundation-Sec-8B-Instruct-hereticI1-Q3_K_S8.0B3.41 GiB1.13 GiB5.58 GiB0.00 GiB36±22%
dolphin-2.9-llama3-8bQ3_K_S8.0B3.41 GiB1.13 GiB5.58 GiB0.00 GiB36±22%
saiga_llama3_8bQ3_K_S8.0B3.41 GiB1.13 GiB5.58 GiB0.00 GiB36±22%
Meta-Llama-3-8B-InstructQ3_K_S8.0B3.41 GiB1.13 GiB5.58 GiB0.00 GiB36±22%
MiniCPM-Llama3-V-2_5Q3_K_S8.5B3.41 GiB1.13 GiB5.58 GiB0.00 GiB36±22%
llama-3-8b-bnb-4bitQ3_K_S8.2B3.41 GiB1.13 GiB5.58 GiB0.00 GiB36±22%
Llama3-ChatQA-1.5-8BQ3_K_S8.0B3.41 GiB1.13 GiB5.58 GiB0.00 GiB36±22%
qwen-indic-v1I1-IQ3_S7.6B3.28 GiB1.27 GiB5.58 GiB0.00 GiB36±22%
Meta-Llama-3-8BQ3_K_S8.0B3.41 GiB1.13 GiB5.58 GiB0.00 GiB36±22%
Ministral-8B-Instruct-2410IQ3_XS8.0B3.28 GiB1.27 GiB5.58 GiB0.00 GiB36±22%
Llama-3.1-Tulu-3-8BQ3_K_S8.0B3.41 GiB1.13 GiB5.58 GiB0.00 GiB36±22%
Llama-3-Groq-8B-Tool-UseQ3_K_S8.0B3.41 GiB1.13 GiB5.58 GiB0.00 GiB36±22%
llama3.1-heretic-unsensoredI1-Q3_K_S8.0B3.41 GiB1.13 GiB5.58 GiB0.00 GiB36±22%
Dolphin3.0-Llama3.1-8BQ3_K_S8.0B3.41 GiB1.13 GiB5.58 GiB0.00 GiB36±22%
dolphin-2.9.4-llama3.1-8bQ3_K_S8.0B3.41 GiB1.13 GiB5.58 GiB0.00 GiB36±22%
Dolphin3.0-Llama3.1-8B-abliteratedQ3_K_S8.0B3.41 GiB1.13 GiB5.58 GiB0.00 GiB36±22%
Anubis-Mini-8B-v1I1-Q3_K_S8.0B3.41 GiB1.13 GiB5.58 GiB0.00 GiB36±22%
Deepseek-R1-Distill-NSFW-RPv1Q3_K_S8.0B3.41 GiB1.13 GiB5.58 GiB0.00 GiB36±22%
LLAMA-3_8B_Unaligned_BETAQ3_K_S8.0B3.41 GiB1.13 GiB5.58 GiB0.00 GiB36±22%
Llama-3.1-8BQ3_K_S8.0B3.41 GiB1.13 GiB5.58 GiB0.00 GiB36±22%
Meta-Llama-3-8BQ3_K_S8.0B3.41 GiB1.13 GiB5.58 GiB0.00 GiB36±22%
L3.1-Dark-Reasoning-LewdPlay-evo-Hermes-R1-Uncensored-8B-hereticI1-Q3_K_S8.0B3.41 GiB1.13 GiB5.58 GiB0.00 GiB36±22%
L3.1-Dark-Reasoning-LewdPlay-evo-Hermes-R1-Uncensored-8BI1-Q3_K_S8.0B3.41 GiB1.13 GiB5.58 GiB0.00 GiB36±22%
Gluon-8BI1-Q3_K_S8.0B3.41 GiB1.13 GiB5.58 GiB0.00 GiB36±22%
Llama3.3-8B-Instruct-Thinking-Heretic-Uncensored-Claude-4.5-Opus-High-ReasoningI1-Q3_K_S8.0B3.41 GiB1.13 GiB5.58 GiB0.00 GiB36±22%
Llama3.3-8B-Instruct-Thinking-Claude-4.5-Opus-High-ReasoningI1-Q3_K_S8.0B3.41 GiB1.13 GiB5.58 GiB0.00 GiB36±22%
Hypnos-i1-8BQ3_K_S8.0B3.41 GiB1.13 GiB5.58 GiB0.00 GiB36±22%
grok-oss-Apollyon-8B-hereticI1-Q3_K_S8.0B3.41 GiB1.13 GiB5.58 GiB0.00 GiB36±22%
Llama-3.3-8B-Instruct-128K-JbliteratedI1-Q3_K_S8.0B3.41 GiB1.13 GiB5.58 GiB0.00 GiB36±22%
PsyCoPref-Llama3-8BI1-Q3_K_S8.0B3.41 GiB1.13 GiB5.58 GiB0.00 GiB36±22%
Llama3.1-GptDeluxe-8BI1-Q3_K_S8.0B3.41 GiB1.13 GiB5.58 GiB0.00 GiB36±22%
SelfCite-8B-CC-SFTI1-Q3_K_S8.0B3.41 GiB1.13 GiB5.58 GiB0.00 GiB36±22%
ai-girlfriend-v2I1-Q3_K_S8.0B3.41 GiB1.13 GiB5.58 GiB0.00 GiB36±22%
Llama-3.3-8B-Instruct-128K_AbliteratedI1-Q3_K_S8.0B3.41 GiB1.13 GiB5.58 GiB0.00 GiB36±22%
Aisha-Uncensored-v2I1-Q3_K_S8.0B3.41 GiB1.13 GiB5.58 GiB0.00 GiB36±22%
grok-oss-Apollyon-8BI1-Q3_K_S8.0B3.41 GiB1.13 GiB5.58 GiB0.00 GiB36±22%
HARC-Llama-3.1-8B-InstructI1-Q3_K_S8.0B3.41 GiB1.13 GiB5.58 GiB0.00 GiB36±22%
Lumen-1.2.5I1-Q3_K_S8.0B3.41 GiB1.13 GiB5.58 GiB0.00 GiB36±22%
Llama-3.3-8B-Instruct-128KI1-Q3_K_S8.0B3.41 GiB1.13 GiB5.58 GiB0.00 GiB36±22%
Stoicism1_Llama3.1-8b-instructI1-Q3_K_S8.0B3.41 GiB1.13 GiB5.58 GiB0.00 GiB36±22%
grok-oss-Thanatos-8BI1-Q3_K_S8.0B3.41 GiB1.13 GiB5.58 GiB0.00 GiB36±22%
Llama-3.1-8B-Lexi-Uncensored-V2Q3_K_S8.0B3.41 GiB1.13 GiB5.58 GiB0.00 GiB36±22%
Llama-3.1-8B-InstructQ3_K_S8.0B3.41 GiB1.13 GiB5.58 GiB0.00 GiB36±22%
L3-8B-Sunfall-v0.5-Stheno-v3.2Q3_K_S8.0B3.41 GiB1.13 GiB5.58 GiB0.00 GiB36±22%
OpenElla-NovelWriter-Requiem-AscendedI1-Q3_K_S8.0B3.41 GiB1.13 GiB5.58 GiB0.00 GiB36±22%
Llama-3.1-Nemotron-Nano-8B-v1Q3_K_S8.0B3.41 GiB1.13 GiB5.58 GiB0.00 GiB36±22%
DeepSeek-R1-Distill-Llama-8BQ3_K_S8.0B3.41 GiB1.13 GiB5.58 GiB0.00 GiB36±22%
llama-joycaption-beta-one-hf-llavaI1-Q3_K_S8.5B3.41 GiB1.13 GiB5.58 GiB0.00 GiB36±22%
Llama-3.1-Swallow-8B-Instruct-v0.5Q3_K_S8.0B3.41 GiB1.13 GiB5.58 GiB0.00 GiB36±22%
DarkIdol-Llama-3.1-8B-Instruct-1.3-UncensoredI1-Q3_K_S8.0B3.41 GiB1.13 GiB5.58 GiB0.00 GiB36±22%
calme-2.3-legalkit-8bI1-Q3_K_S8.0B3.41 GiB1.13 GiB5.58 GiB0.00 GiB36±22%
Llama3.1-DilemmaDeluxe-8BI1-Q3_K_S8.0B3.41 GiB1.13 GiB5.58 GiB0.00 GiB36±22%
Llama-3.1-8B-Stheno-v3.4I1-Q3_K_S8.0B3.41 GiB1.13 GiB5.58 GiB0.00 GiB36±22%
DarkIdol-Llama-3.1-8B-Instruct-1.0-UncensoredI1-Q3_K_S8.0B3.41 GiB1.13 GiB5.58 GiB0.00 GiB36±22%
DarkIdol-Llama-3.1-8B-Instruct-1.2-UncensoredQ3_K_S8.0B3.41 GiB1.13 GiB5.58 GiB0.00 GiB36±22%
SzilviaB-Daredevil-LongWriter-8B_abliteratedI1-Q3_K_S8.0B3.41 GiB1.13 GiB5.58 GiB0.00 GiB36±22%
Llama-3.1-SuperNova-LiteI1-Q3_K_S8.0B3.41 GiB1.13 GiB5.58 GiB0.00 GiB36±22%
Humanish-Roleplay-Llama-3.1-8BI1-Q3_K_S8.0B3.41 GiB1.13 GiB5.58 GiB0.00 GiB36±22%
Llama-3.1_OpenScholar-8BI1-Q3_K_S8.0B3.41 GiB1.13 GiB5.58 GiB0.00 GiB36±22%
●From the file◐Predictedwhat these mean

Speed is modeled, not measured: decode is memory-bandwidth bound, so tokens per second is bytes read per token against achievable bandwidth. Mixture-of-experts models carry a wider band because only the routed experts are read each step, and few have been measured publicly.

Questions people ask

What AI models can a RTX A2000 run?
1085 of 2118 indexed open-weight models fit a RTX A2000 at 32,768 context with q4_0 KV cache, the largest being Foundation-Sec-8B-Instruct at I1-Q3_K_S. That covers text, vision-language, image, video and speech models.
How much usable memory does a RTX A2000 actually have?
Its nameplate is 6 GB, but about 5.58 GiB is available to a model once driver and compositor overhead is accounted for.
Is a RTX A2000 fast for local AI?
Its memory bandwidth is 288 GB/s, and that figure — not teraflops — is what governs token generation speed. Capacity decides what you can run; bandwidth decides how fast it runs.