NVIDIA · workstation

RTX A2000

RTX A2000 has 6 GB of VRAM at 288 GB/s — about 5.58 GiB usable after driver and compositor overhead. 1173 of 2118 indexed models fit at 8K context with q8_0 KV.

Spec sheet· bandwidth, theoreticalFrom the file· fit from summed bytesPredicted· speed
Memory
6 GB
GDDR6
Bandwidth
288 GB/s
192-bit bus
Tensor FP16
32 TF
dense
TDP
70 W
$449 MSRP
Context4K8K16K32K64K128K
KV cachef16q8_0q4_0quantizing the KV cache is a ~2× lever on the dominant term at long context
text 998vision language 89embedding 26image 1audio asr 38audio tts 19video 2

What fits at 8K context

largest quantization that fits, per model · 1173 of 2118 indexed
ModelBest quantParamsWeights●KV●Total in memory◐Headroom◐tok/s◐
LFM2.5-Queen-Opus-4.7-8B-A1BMoEI1-Q4_K_S8.5B4.53 GiB0.05 GiB5.58 GiB0.00 GiB104±37%
LFM2.5-8B-A1B-KO-SFTMoEI1-Q4_K_S8.5B4.53 GiB0.05 GiB5.58 GiB0.00 GiB104±37%
LFM2.5-8B-A1B-hereticMoEI1-Q4_K_S8.5B4.53 GiB0.05 GiB5.58 GiB0.00 GiB104±37%
LFM2.5-8B-A1B-SOMPOA-heresyMoEI1-Q4_K_S8.5B4.53 GiB0.05 GiB5.58 GiB0.00 GiB104±37%
Huihui-LFM2.5-8B-A1B-abliteratedMoEI1-Q4_K_S8.5B4.53 GiB0.05 GiB5.58 GiB0.00 GiB104±37%
Supertron2.1-8B-A1BMoEI1-Q4_K_S8.5B4.53 GiB0.05 GiB5.58 GiB0.00 GiB104±37%
gemma-3-12b-it-vl-Gemini-3-Pro-Preview-Heretic-Uncensored-ThinkingI1-IQ2_M12.2B4.01 GiB0.51 GiB5.57 GiB0.01 GiB36±22%
gemma-3-12b-it-vl-Deepseek-v3.1-Heretic-Uncensored-ThinkingI1-IQ2_M12.2B4.01 GiB0.51 GiB5.57 GiB0.01 GiB36±22%
gemma-3-12b-it-vl-GLM-4.7-Flash-Heretic-Uncensored-ThinkingI1-IQ2_M12.2B4.01 GiB0.51 GiB5.57 GiB0.01 GiB36±22%
Floppa-12B-Gemma3-UncensoredI1-IQ2_M12.2B4.01 GiB0.51 GiB5.57 GiB0.01 GiB36±22%
gemma-3-12b-it-hereticI1-IQ2_M12.2B4.01 GiB0.51 GiB5.57 GiB0.01 GiB36±22%
gemma-3-12b-it-abliteratedIQ2_M12.2B4.01 GiB0.51 GiB5.57 GiB0.01 GiB36±22%
Qwen3-VL-8B-Instruct-HereticI1-IQ1_S8.8B3.94 GiB0.60 GiB5.57 GiB0.01 GiB36±22%
qwen-indic-v1I1-IQ4_XS7.6B3.94 GiB0.60 GiB5.57 GiB0.01 GiB36±22%
MARTHA-LXVII.8B_QWEN-3.5-3.6_prune_9b-3.6_baseIQ4_XS8.1B4.42 GiB0.12 GiB5.57 GiB0.01 GiB36±22%
Tini-Cybersec-8B-A1BMoEIQ4_NL8.5B4.52 GiB0.05 GiB5.57 GiB0.01 GiB104±37%
INTELLECT-1-InstructI1-IQ3_XXS10.2B3.83 GiB0.70 GiB5.57 GiB0.01 GiB36±22%
salamandra-7b-instruct-2606I1-Q3_K_L7.8B4.00 GiB0.53 GiB5.57 GiB0.01 GiB36±22%
Fara1.5-9BIQ3_M9.4B4.40 GiB0.13 GiB5.57 GiB0.01 GiB36±22%
QwenPaw-Flash-9BIQ3_M9.4B4.40 GiB0.13 GiB5.57 GiB0.01 GiB36±22%
grug-9bIQ3_M9.4B4.40 GiB0.13 GiB5.57 GiB0.01 GiB36±22%
OmniCoder-9BIQ3_M9.4B4.40 GiB0.13 GiB5.57 GiB0.01 GiB36±22%
Ornith-1.0-9BIQ3_M9.2B4.40 GiB0.13 GiB5.57 GiB0.01 GiB36±22%
Qwen3.5-9B-NeoIQ3_M9.7B4.40 GiB0.13 GiB5.57 GiB0.01 GiB36±22%
Forsaken-Void-12BI1-IQ2_S12.2B3.85 GiB0.66 GiB5.57 GiB0.01 GiB36±22%
Silver-Siren-ST-12BI1-IQ2_S12.2B3.85 GiB0.66 GiB5.57 GiB0.01 GiB36±22%
Tess-3-Mistral-Nemo-12BI1-IQ2_S12.2B3.85 GiB0.66 GiB5.57 GiB0.01 GiB36±22%
KrakenSakura-Maelstrom-12B-v1IQ2_S12.2B3.85 GiB0.66 GiB5.57 GiB0.01 GiB36±22%
Dans-PersonalityEngine-V1.3.0-12bI1-IQ2_S12.2B3.85 GiB0.66 GiB5.57 GiB0.01 GiB36±22%
MN-12B-Runeweaver-RP-RUI1-IQ2_S12.2B3.85 GiB0.66 GiB5.57 GiB0.01 GiB36±22%
Impish_Bloodmoon_12BI1-IQ2_S12.2B3.85 GiB0.66 GiB5.57 GiB0.01 GiB36±22%
Wayfarer-2-12BI1-IQ2_S12.2B3.85 GiB0.66 GiB5.57 GiB0.01 GiB36±22%
Wayfarer-12BI1-IQ2_S12.2B3.85 GiB0.66 GiB5.57 GiB0.01 GiB36±22%
Muse-12BI1-IQ2_S12.2B3.85 GiB0.66 GiB5.57 GiB0.01 GiB36±22%
writing-roleplay-20k-context-nemo-12b-v1.0IQ2_S12.2B3.85 GiB0.66 GiB5.57 GiB0.01 GiB36±22%
mini-magnum-12b-v1.1IQ2_S12.2B3.85 GiB0.66 GiB5.57 GiB0.01 GiB36±22%
MN-Violet-Lotus-12BI1-IQ2_S12.2B3.85 GiB0.66 GiB5.57 GiB0.01 GiB36±22%
Rocinante-X-12B-v1-Heretic-UncensoredI1-IQ2_S12.2B3.85 GiB0.66 GiB5.57 GiB0.01 GiB36±22%
Mistral-Heretica-12BI1-IQ2_S12.2B3.85 GiB0.66 GiB5.57 GiB0.01 GiB36±22%
Lumimaid-Magnum-v4-12BIQ2_S12.2B3.85 GiB0.66 GiB5.57 GiB0.01 GiB36±22%
arcee-fusion-lumaid-12BI1-IQ2_S12.2B3.85 GiB0.66 GiB5.57 GiB0.01 GiB36±22%
Mistral-NeMo-12B-AbliteratedI1-IQ2_S12.2B3.85 GiB0.66 GiB5.57 GiB0.01 GiB36±22%
Captain-Eris_Violet-V0.420-12BI1-IQ2_S12.2B3.85 GiB0.66 GiB5.57 GiB0.01 GiB36±22%
Rocinante-X-12B-v1-absolute-heresyI1-IQ2_S12.2B3.85 GiB0.66 GiB5.57 GiB0.01 GiB36±22%
Rocinante-X-12B-v1I1-IQ2_S12.2B3.85 GiB0.66 GiB5.57 GiB0.01 GiB36±22%
MN-12b-RP-InkIQ2_S12.2B3.85 GiB0.66 GiB5.57 GiB0.01 GiB36±22%
magnum-v4-12bIQ2_S12.2B3.85 GiB0.66 GiB5.57 GiB0.01 GiB36±22%
Mistral-Nemo-2407-12B-Thinking-Claude-Gemini-GPT5.2-Uncensored-HERETICI1-IQ2_S12.2B3.85 GiB0.66 GiB5.57 GiB0.01 GiB36±22%
Dans-SakuraKaze-V1.0.0-12bI1-IQ2_S12.2B3.85 GiB0.66 GiB5.57 GiB0.01 GiB36±22%
Mistral-Nemo-Inst-2407-12B-Thinking-Uncensored-HERETIC-HI-Claude-OpusI1-IQ2_S12.2B3.85 GiB0.66 GiB5.57 GiB0.01 GiB36±22%
Mistral-Nemo-Instruct-2407-12B-Thinking-M-Claude-Opus-High-ReasoningI1-IQ2_S12.2B3.85 GiB0.66 GiB5.57 GiB0.01 GiB36±22%
Mordant-12B-ThinkI1-IQ2_S12.2B3.85 GiB0.66 GiB5.57 GiB0.01 GiB36±22%
MN-12B-Mag-Mell-R1IQ2_S12.2B3.85 GiB0.66 GiB5.57 GiB0.01 GiB36±22%
Riverfish-Rocinante-12B-SFT-DPOI1-IQ2_S12.2B3.85 GiB0.66 GiB5.57 GiB0.01 GiB36±22%
MN-Violet-Lotus-12B-HereticI1-IQ2_S12.2B3.85 GiB0.66 GiB5.57 GiB0.01 GiB36±22%
Himeyuri-Magnum-12B-HereticMergeI1-IQ2_S12.2B3.85 GiB0.66 GiB5.57 GiB0.01 GiB36±22%
Kinggaroo-12b-v1I1-IQ2_S12.2B3.85 GiB0.66 GiB5.57 GiB0.01 GiB36±22%
Magnum-Picaro-0.7-v2-12bI1-IQ2_S12.2B3.85 GiB0.66 GiB5.57 GiB0.01 GiB36±22%
magnum-v2.5-12b-ktoI1-IQ2_S12.2B3.85 GiB0.66 GiB5.57 GiB0.01 GiB36±22%
Mistral-Nemo-Prism-12BI1-IQ2_S12.2B3.85 GiB0.66 GiB5.57 GiB0.01 GiB36±22%
●From the file◐Predictedwhat these mean

Speed is modeled, not measured: decode is memory-bandwidth bound, so tokens per second is bytes read per token against achievable bandwidth. Mixture-of-experts models carry a wider band because only the routed experts are read each step, and few have been measured publicly.

Questions people ask

What AI models can a RTX A2000 run?
1173 of 2118 indexed open-weight models fit a RTX A2000 at 8,192 context with q8_0 KV cache, the largest being LFM2.5-Queen-Opus-4.7-8B-A1B at I1-Q4_K_S. That covers text, vision-language, image, video and speech models.
How much usable memory does a RTX A2000 actually have?
Its nameplate is 6 GB, but about 5.58 GiB is available to a model once driver and compositor overhead is accounted for.
Is a RTX A2000 fast for local AI?
Its memory bandwidth is 288 GB/s, and that figure — not teraflops — is what governs token generation speed. Capacity decides what you can run; bandwidth decides how fast it runs.
RTX A2000 — what AI models can it run locally? — ossmodeldb