NVIDIA · datacenter

Tesla V100 16GB

Tesla V100 16GB has 16 GB of VRAM at 900 GB/s — about 14.88 GiB usable after driver and compositor overhead. 1813 of 2118 indexed models fit at 32K context with q8_0 KV.

Spec sheet· bandwidth, theoreticalFrom the file· fit from summed bytesPredicted· speed
Memory
16 GB
HBM2
Bandwidth
900 GB/s
4096-bit bus
Tensor FP16
125 TF
dense
TDP
300 W
Context4K8K16K32K64K128K
KV cachef16q8_0q4_0quantizing the KV cache is a ~2× lever on the dominant term at long context
text 1551vision language 159video 15audio asr 39image 2embedding 26audio tts 21

What fits at 32K context

largest quantization that fits, per model · 1813 of 2118 indexed
ModelBest quantParamsWeights●KV●Total in memory◐Headroom◐tok/s◐
ERNIE-4.5-21B-A3B-ThinkingQ4_121.8B12.93 GiB0.93 GiB14.88 GiB0.00 GiB37±22%
ERNIE-4.5-21B-A3B-PTQ4_121.9B12.93 GiB0.93 GiB14.88 GiB0.00 GiB37±22%
Gemma-4-Novelist-Eclipse-31BIQ2_XXS32.7B10.52 GiB3.28 GiB14.88 GiB0.00 GiB37±22%
Gemma-4-31B-StyleTuneIQ2_XXS32.7B10.52 GiB3.28 GiB14.88 GiB0.00 GiB37±22%
Qwen3-16B-A3BMoEQ6_K16.0B12.28 GiB1.59 GiB14.87 GiB0.01 GiB75±37%
solar-pro-preview-instructKV unresolvedIQ3_XS22.1B8.50 GiB5.31 GiB14.87 GiB0.01 GiB37±22%
Llama3.2-24B-A3B-II-Dark-Champion-INSTRUCT-Heretic-Abliterated-UncensoredMoEI1-Q5_K_M18.0B12.00 GiB1.86 GiB14.87 GiB0.01 GiB71±37%
granite-20b-code-instruct-8kQ5_K_M20.1B13.79 GiB0.00 GiB14.87 GiB0.01 GiB37±22%
granite-20b-code-base-8kI1-Q5_K_M20.1B13.79 GiB0.00 GiB14.87 GiB0.01 GiB37±22%
internlm2-math-plus-20bI1-Q4_K_S19.9B10.62 GiB3.19 GiB14.87 GiB0.01 GiB37±22%
granite-34b-code-base-8kI1-IQ3_S33.7B13.79 GiB0.00 GiB14.87 GiB0.01 GiB37±22%
GLM-4.7-Flash-REAP-23B-A3B-absolute-heresyMoEI1-Q4_K_M23.0B12.98 GiB0.88 GiB14.86 GiB0.02 GiB104±37%
codegeex4-all-9bIQ2_XXS9.4B3.19 GiB10.63 GiB14.86 GiB0.02 GiB37±22%
glm-4-9b-chatIQ2_XXS9.4B3.19 GiB10.63 GiB14.86 GiB0.02 GiB37±22%
Skywork-R1V3-38BIQ3_M38.4B13.79 GiB0.00 GiB14.86 GiB0.02 GiB37±22%
Nemotron-Mini-4B-InstructQ6_K4.2B11.72 GiB2.13 GiB14.86 GiB0.02 GiB37±22%
dolphin-2.6-mixtral-8x7bMoEI1-IQ2_XXS46.7B11.69 GiB2.13 GiB14.86 GiB0.02 GiB51±37%
xLAM-8x7b-rMoEIQ2_XXS46.7B11.69 GiB2.13 GiB14.86 GiB0.02 GiB51±37%
Llama-3.2-3BF163.2B11.98 GiB1.86 GiB14.85 GiB0.03 GiB37±22%
grug-27bIQ3_M27.4B12.73 GiB1.06 GiB14.85 GiB0.03 GiB37±22%
Carnice-V2-27bIQ3_M27.4B12.73 GiB1.06 GiB14.85 GiB0.03 GiB37±22%
Fara1.5-27BIQ3_M27.4B12.73 GiB1.06 GiB14.85 GiB0.03 GiB37±22%
SOLAR-10.7B-Instruct-v1.0-uncensoredQ8_010.7B10.62 GiB3.19 GiB14.85 GiB0.03 GiB37±22%
Nous-Hermes-2-SOLAR-10.7BQ8_010.7B10.62 GiB3.19 GiB14.85 GiB0.03 GiB37±22%
SOLAR-10.7B-Instruct-v1.0Q8_010.7B10.62 GiB3.19 GiB14.85 GiB0.03 GiB37±22%
Dolphin3.0-Llama3.2-3BF323.2B11.98 GiB1.86 GiB14.84 GiB0.04 GiB37±22%
Hermes-3-Llama-3.2-3BF323.2B11.98 GiB1.86 GiB14.84 GiB0.04 GiB37±22%
v6-Finch-7B-HFQ5_K_M7.6B5.29 GiB8.50 GiB14.84 GiB0.04 GiB37±22%
rwkv-6-world-7bQ5_K_M7.6B5.29 GiB8.50 GiB14.84 GiB0.04 GiB37±22%
gemma-4-19B-A4B-it-INSTRUCT-Heretic-UncensoredMoEI1-Q5_K_M19.0B13.03 GiB0.82 GiB14.83 GiB0.05 GiB37±22%
gemma-4-19B-A4B-it-The-DECKARD-Heretic-Uncensored-ThinkingMoEI1-Q5_K_M19.0B13.03 GiB0.82 GiB14.83 GiB0.05 GiB37±22%
gemma-4-19b-a4b-it-REAP-hereticMoEI1-Q5_K_M19.0B13.03 GiB0.82 GiB14.83 GiB0.05 GiB37±22%
Gemma-4-19BMoEI1-Q5_K_M19.0B13.03 GiB0.82 GiB14.83 GiB0.05 GiB37±22%
SambaLingo-Japanese-ChatI1-Q6_K6.9B5.31 GiB8.50 GiB14.83 GiB0.05 GiB37±22%
Carnice-Qwen3.6-MoE-35B-A3BMoEI1-IQ3_XS36.0B13.49 GiB0.33 GiB14.83 GiB0.05 GiB173±37%
Qwen35B-Agent-R2-AbliteratedMoEI1-IQ3_XS34.7B13.49 GiB0.33 GiB14.83 GiB0.05 GiB173±37%
Qwen3.6-35B-A3B-Claude-4.6-Opus-Reasoning-DistilledMoEI1-IQ3_XS36.0B13.49 GiB0.33 GiB14.83 GiB0.05 GiB173±37%
Darwin-35B-A3B-OpusMoEI1-IQ3_XS36.0B13.49 GiB0.33 GiB14.83 GiB0.05 GiB173±37%
Qwen35B-Agent-R2MoEI1-IQ3_XS34.7B13.49 GiB0.33 GiB14.83 GiB0.05 GiB173±37%
Carnice-MoE-35B-A3BMoEI1-IQ3_XS36.0B13.49 GiB0.33 GiB14.83 GiB0.05 GiB173±37%
spoomplesmaxx-flash-35B-A3MoEI1-IQ3_XS35.1B13.49 GiB0.33 GiB14.83 GiB0.05 GiB173±37%
Huihui-Qwen3.6-35B-A3B-Claude-4.7-Opus-abliteratedMoEI1-IQ3_XS36.0B13.49 GiB0.33 GiB14.83 GiB0.05 GiB173±37%
Qwen3.6-35B-A3B-Uncensored-AggressiveMoEI1-IQ3_XS35.1B13.49 GiB0.33 GiB14.83 GiB0.05 GiB173±37%
WorldSim-Opus-3.6-35B-A3BMoEI1-IQ3_XS35.1B13.49 GiB0.33 GiB14.83 GiB0.05 GiB173±37%
Qwen3.6-35B-A3B-abliterated-MAXMoEI1-IQ3_XS35.1B13.49 GiB0.33 GiB14.83 GiB0.05 GiB173±37%
Huihui-Qwen3.6-35B-A3B-abliteratedMoEI1-IQ3_XS36.0B13.49 GiB0.33 GiB14.83 GiB0.05 GiB173±37%
Qwopus3.6-35B-A3B-v1MoEI1-IQ3_XS36.0B13.49 GiB0.33 GiB14.83 GiB0.05 GiB173±37%
Qwen3.6-35B-A3B-StyleTuneMoEI1-IQ3_XS35.1B13.49 GiB0.33 GiB14.83 GiB0.05 GiB173±37%
Qwen3.6-35B-A3B-abliteratedMoEI1-IQ3_XS35.1B13.49 GiB0.33 GiB14.83 GiB0.05 GiB173±37%
0GM-1.0-35B-A3B-0427MoEI1-IQ3_XS36.0B13.49 GiB0.33 GiB14.83 GiB0.05 GiB173±37%
GLM-4-32B-0414-Korean-CultureI1-IQ3_XS32.6B12.72 GiB1.01 GiB14.83 GiB0.05 GiB37±22%
GLM-Z1-32B-0414IQ3_XS32.6B12.72 GiB1.01 GiB14.83 GiB0.05 GiB37±22%
GLM-4-32B-0414IQ3_XS32.6B12.72 GiB1.01 GiB14.83 GiB0.05 GiB37±22%
LFM2-24B-A2BMoEQ4_K_L23.8B13.47 GiB0.33 GiB14.82 GiB0.06 GiB140±37%
Kimi-VL-A3B-InstructMoEI1-Q6_K16.4B13.30 GiB0.50 GiB14.82 GiB0.06 GiB112±37%
Kimi-VL-A3B-Thinking-2506MoEQ6_K16.4B13.30 GiB0.50 GiB14.82 GiB0.06 GiB112±37%
Moonlight-16B-A3B-InstructMoEQ6_K16.0B13.30 GiB0.50 GiB14.82 GiB0.06 GiB112±37%
North-Mini-Code-1.0MoEUD-Q3_K_M30.5B13.24 GiB0.60 GiB14.81 GiB0.07 GiB126±37%
Seed-OSS-36B-InstructUD-IQ2_XXS36.2B9.46 GiB4.25 GiB14.81 GiB0.07 GiB37±22%
Skyfall-31B-v4.2IQ2_S31.4B10.10 GiB3.59 GiB14.81 GiB0.07 GiB37±22%
●From the file◐Predictedwhat these mean

Speed is modeled, not measured: decode is memory-bandwidth bound, so tokens per second is bytes read per token against achievable bandwidth. Mixture-of-experts models carry a wider band because only the routed experts are read each step, and few have been measured publicly.

Questions people ask

What AI models can a Tesla V100 16GB run?
1813 of 2118 indexed open-weight models fit a Tesla V100 16GB at 32,768 context with q8_0 KV cache, the largest being ERNIE-4.5-21B-A3B-Thinking at Q4_1. That covers text, vision-language, image, video and speech models.
How much usable memory does a Tesla V100 16GB actually have?
Its nameplate is 16 GB, but about 14.88 GiB is available to a model once driver and compositor overhead is accounted for.
Is a Tesla V100 16GB fast for local AI?
Its memory bandwidth is 900 GB/s, and that figure — not teraflops — is what governs token generation speed. Capacity decides what you can run; bandwidth decides how fast it runs.