NVIDIA · workstation

RTX A1000

RTX A1000 has 8 GB of VRAM at 192 GB/s — about 7.44 GiB usable after driver and compositor overhead. 590 of 2118 indexed models fit at 128K context with q8_0 KV.

Spec sheet· bandwidth, theoreticalFrom the file· fit from summed bytesPredicted· speed
Memory
8 GB
GDDR6
Bandwidth
192 GB/s
128-bit bus
Tensor FP16
27 TF
dense
TDP
50 W
$365 MSRP
Context4K8K16K32K64K128K
KV cachef16q8_0q4_0quantizing the KV cache is a ~2× lever on the dominant term at long context
text 449vision language 69embedding 15audio asr 31video 7image 1audio tts 18

What fits at 128K context

largest quantization that fits, per model · 590 of 2118 indexed
ModelBest quantParamsWeights●KV●Total in memory◐Headroom◐tok/s◐
Falcon3-1B-InstructQ8_01.7B1.66 GiB4.78 GiB7.44 GiB0.00 GiB17±22%
Qwen2.5-VL-7B-InstructUD-IQ2_M8.3B2.66 GiB3.72 GiB7.43 GiB0.01 GiB17±22%
FrickFritz-4BQ8_04.7B4.29 GiB2.13 GiB7.43 GiB0.01 GiB17±22%
Newton-bot-3-VLM-mini-4BQ8_04.7B4.29 GiB2.13 GiB7.43 GiB0.01 GiB17±22%
qwen3.5-4b-agentic-coder-v4Q8_04.7B4.29 GiB2.13 GiB7.43 GiB0.01 GiB17±22%
Myth-4BQ8_04.3B4.29 GiB2.13 GiB7.43 GiB0.01 GiB17±22%
Qwen3.5-4B-UncensoredQ8_04.7B4.29 GiB2.13 GiB7.43 GiB0.01 GiB17±22%
JOSIE-2-4B-PreviewQ8_04.7B4.29 GiB2.13 GiB7.43 GiB0.01 GiB17±22%
Surogate-3.5-4BQ8_05.3B4.29 GiB2.13 GiB7.43 GiB0.01 GiB17±22%
Qwopus3.5-4B-v3Q8_04.7B4.29 GiB2.13 GiB7.43 GiB0.01 GiB17±22%
granite-3.1-2b-instructIQ3_M2.5B1.11 GiB5.31 GiB7.43 GiB0.01 GiB17±22%
granite-3.1-3b-a800m-instructMoEQ5_K_L3.3B2.21 GiB4.25 GiB7.43 GiB0.01 GiB13±37%
GLM-4.6V-FlashQ2_K10.3B3.73 GiB2.66 GiB7.43 GiB0.01 GiB17±22%
GLM-Z1-9B-0414Q2_K9.4B3.73 GiB2.66 GiB7.43 GiB0.01 GiB17±22%
glm4.1v-9b-base-sftI1-Q2_K10.3B3.73 GiB2.66 GiB7.43 GiB0.01 GiB17±22%
GLM-4-9B-0414Q2_K9.4B3.73 GiB2.66 GiB7.43 GiB0.01 GiB17±22%
GLM-4.1V-9B-ThinkingQ2_K10.3B3.73 GiB2.66 GiB7.43 GiB0.01 GiB17±22%
internlm3-8b-instructQ2_K8.8B3.21 GiB3.19 GiB7.43 GiB0.01 GiB17±22%
GrammarCoder-7B-BaseI1-Q2_K_S7.6B2.65 GiB3.72 GiB7.42 GiB0.02 GiB17±22%
MARTHA-LXVII.8B_QWEN-3.5-3.6_prune_9b-3.6_baseI1-Q4_K_S8.1B4.52 GiB1.86 GiB7.42 GiB0.02 GiB17±22%
Tini-Cybersec-8B-A1BMoEQ5_K_M8.5B5.62 GiB0.80 GiB7.42 GiB0.02 GiB34±37%
nomic-embed-codeQ2_K7.1B2.64 GiB3.72 GiB7.42 GiB0.02 GiB17±22%
DeepHat-V1-7B-Heretic-AbliteratedI1-Q2_K_S7.6B2.64 GiB3.72 GiB7.41 GiB0.03 GiB17±22%
ShizhenGPT-7B-VLI1-Q2_K_S8.3B2.64 GiB3.72 GiB7.41 GiB0.03 GiB17±22%
HuatuoGPT-o1-7BI1-Q2_K_S7.6B2.64 GiB3.72 GiB7.41 GiB0.03 GiB17±22%
MathSmith-DS-Qwen-7B-LongCoTI1-Q2_K_S7.6B2.64 GiB3.72 GiB7.41 GiB0.03 GiB17±22%
AstraGPTCoder-7BI1-Q2_K_S7.6B2.64 GiB3.72 GiB7.41 GiB0.03 GiB17±22%
Qwen2.5-Coder-7B-Instruct-Ghidra-v2I1-Q2_K_S7.6B2.64 GiB3.72 GiB7.41 GiB0.03 GiB17±22%
EsDrac-v1-7BI1-Q2_K_S7.6B2.64 GiB3.72 GiB7.41 GiB0.03 GiB17±22%
Hemlock-Apothecary-7B-GRPO-e3I1-Q2_K_S7.6B2.64 GiB3.72 GiB7.41 GiB0.03 GiB17±22%
openhands-lm-7b-v0.1I1-Q2_K_S7.6B2.64 GiB3.72 GiB7.41 GiB0.03 GiB17±22%
Hemlock2-Coder-7B-GRPOI1-Q2_K_S7.6B2.64 GiB3.72 GiB7.41 GiB0.03 GiB17±22%
shellwhiz-7bI1-Q2_K_S7.6B2.64 GiB3.72 GiB7.41 GiB0.03 GiB17±22%
Qwen2.5-Coder-7B-Instruct-OBLITERATED-advancedI1-Q2_K_S7.6B2.64 GiB3.72 GiB7.41 GiB0.03 GiB17±22%
Qwen-STEM-Specialist-7BI1-Q2_K_S7.6B2.64 GiB3.72 GiB7.41 GiB0.03 GiB17±22%
VulnLLM-R-7BI1-Q2_K_S7.6B2.64 GiB3.72 GiB7.41 GiB0.03 GiB17±22%
Garnet-OCR-7B-0422I1-Q2_K_S8.3B2.64 GiB3.72 GiB7.41 GiB0.03 GiB17±22%
UwU-7B-InstructI1-Q2_K_S7.6B2.64 GiB3.72 GiB7.41 GiB0.03 GiB17±22%
Video-R1-7BI1-Q2_K_S8.3B2.64 GiB3.72 GiB7.41 GiB0.03 GiB17±22%
HARC-Qwen2.5-7B-InstructI1-Q2_K_S7.6B2.64 GiB3.72 GiB7.41 GiB0.03 GiB17±22%
Qwen2.5-Coder-7B-AbliteratedI1-Q2_K_S7.6B2.64 GiB3.72 GiB7.41 GiB0.03 GiB17±22%
Bozdogan-7BI1-Q2_K_S7.6B2.64 GiB3.72 GiB7.41 GiB0.03 GiB17±22%
Crazy-AI-ModelI1-Q2_K_S7.6B2.64 GiB3.72 GiB7.41 GiB0.03 GiB17±22%
turbo-ai-7bI1-Q2_K_S7.6B2.64 GiB3.72 GiB7.41 GiB0.03 GiB17±22%
DeepSeek-R1-Distill-Qwen-7B-abliterated-v2I1-Q2_K_S7.6B2.64 GiB3.72 GiB7.41 GiB0.03 GiB17±22%
Ghosty-7BI1-Q2_K_S7.6B2.64 GiB3.72 GiB7.41 GiB0.03 GiB17±22%
SP-7BI1-Q2_K_S7.6B2.64 GiB3.72 GiB7.41 GiB0.03 GiB17±22%
Qwen2.5-Coder-7B-Instruct-UncensoredI1-Q2_K_S7.6B2.64 GiB3.72 GiB7.41 GiB0.03 GiB17±22%
Qwen2.5-VL-7B-Instruct-abliteratedI1-Q2_K_S8.3B2.64 GiB3.72 GiB7.41 GiB0.03 GiB17±22%
DeepSeek-R1-Distill-Qwen-8B-AbliteratedI1-Q2_K_S7.6B2.64 GiB3.72 GiB7.41 GiB0.03 GiB17±22%
Qwen2.5-7BQ2_K_S7.6B2.64 GiB3.72 GiB7.41 GiB0.03 GiB17±22%
Qwen2.5-VL-7B-Instruct-hereticI1-Q2_K_S8.3B2.64 GiB3.72 GiB7.41 GiB0.03 GiB17±22%
AWARES-Qwen2.5-VL-7BI1-Q2_K_S8.3B2.64 GiB3.72 GiB7.41 GiB0.03 GiB17±22%
olmOCR-2-7B-1025I1-Q2_K_S8.3B2.64 GiB3.72 GiB7.41 GiB0.03 GiB17±22%
Med-RwRI1-Q2_K_S8.3B2.64 GiB3.72 GiB7.41 GiB0.03 GiB17±22%
SpatialThinker-7BI1-Q2_K_S8.3B2.64 GiB3.72 GiB7.41 GiB0.03 GiB17±22%
MQ-Coldbrew-BaseI1-Q2_K_S7.6B2.64 GiB3.72 GiB7.41 GiB0.03 GiB17±22%
Kepler-Reasoning-7BI1-Q2_K_S7.6B2.64 GiB3.72 GiB7.41 GiB0.03 GiB17±22%
DeepSeek-R1-Distill-Qwen-7B-Uncensored-ReasonerI1-Q2_K_S7.6B2.64 GiB3.72 GiB7.41 GiB0.03 GiB17±22%
DeepSeek-R1-Distill-Qwen-7B-UncensoredI1-Q2_K_S7.6B2.64 GiB3.72 GiB7.41 GiB0.03 GiB17±22%
●From the file◐Predictedwhat these mean

Speed is modeled, not measured: decode is memory-bandwidth bound, so tokens per second is bytes read per token against achievable bandwidth. Mixture-of-experts models carry a wider band because only the routed experts are read each step, and few have been measured publicly.

Measured on this card

third-party benchmarks, aggregated
Workload◍MedianMiddle 50%Runs
Image generation3.75 it/s3.59–4.057
Benchmarked· n=7

Aggregated from community-submitted runs, so the spread is wide by nature — it covers different models, resolutions, step counts and settings, not one controlled configuration. Read the middle 50% rather than the median alone. These figures are reproduced with attribution from vladmandic-sd-data-benchmark, which publishes no licence — so we display and link rather than redistribute them.

Questions people ask

What AI models can a RTX A1000 run?
590 of 2118 indexed open-weight models fit a RTX A1000 at 131,072 context with q8_0 KV cache, the largest being Falcon3-1B-Instruct at Q8_0. That covers text, vision-language, image, video and speech models.
How much usable memory does a RTX A1000 actually have?
Its nameplate is 8 GB, but about 7.44 GiB is available to a model once driver and compositor overhead is accounted for.
Is a RTX A1000 fast for local AI?
Its memory bandwidth is 192 GB/s, and that figure — not teraflops — is what governs token generation speed. Capacity decides what you can run; bandwidth decides how fast it runs.
RTX A1000 — what AI models can it run locally? — ossmodeldb