NVIDIA · consumer

Titan V

Titan V has 12 GB of VRAM at 651 GB/s — about 11.16 GiB usable after driver and compositor overhead. 1761 of 2118 indexed models fit at 8K context with q8_0 KV.

Spec sheet· bandwidth, theoreticalFrom the file· fit from summed bytesPredicted· speed
Memory
12 GB
HBM2
Bandwidth
651 GB/s
3072-bit bus
Tensor FP16
119 TF
dense
TDP
250 W
$2999 MSRP
Context4K8K16K32K64K128K
KV cachef16q8_0q4_0quantizing the KV cache is a ~2× lever on the dominant term at long context
text 1512video 14vision language 147audio tts 21audio asr 39image 2embedding 26

What fits at 8K context

largest quantization that fits, per model · 1761 of 2118 indexed
ModelBest quantParamsWeights●KV●Total in memory◐Headroom◐tok/s◐
dolphin-2.9.1-mixtral-1x22bMoEI1-IQ3_M22.2B9.37 GiB0.93 GiB11.16 GiB0.00 GiB26±37%
Wan2.1-FLF2V-14B-720PQ4_116.4B10.32 GiB0.00 GiB11.16 GiB0.00 GiB45±12.9%
Wan2.1-I2V-14B-480PQ4_116.4B10.32 GiB0.00 GiB11.15 GiB0.01 GiB45±12.9%
Wan2.1-I2V-14B-720PQ4_116.4B10.32 GiB0.00 GiB11.15 GiB0.01 GiB45±12.9%
Trinity-MiniMoEIQ3_XS26.1B10.23 GiB0.13 GiB11.15 GiB0.01 GiB177±37%
Phi-4-reasoning-plusQ5_K_S14.7B9.45 GiB0.83 GiB11.14 GiB0.02 GiB45±12.9%
Phi-4-reasoningQ5_K_S14.7B9.45 GiB0.83 GiB11.14 GiB0.02 GiB45±12.9%
phi-4Q5_K_S14.7B9.45 GiB0.83 GiB11.14 GiB0.02 GiB45±12.9%
granite-3.1-8b-instructQ4_K8.2B9.65 GiB0.66 GiB11.14 GiB0.02 GiB45±12.9%
Darwin-35B-A3B-OpusMoEIQ2_S36.0B10.25 GiB0.08 GiB11.14 GiB0.02 GiB227±37%
Aurora-Code-1MoEIQ2_S34.7B10.25 GiB0.08 GiB11.14 GiB0.02 GiB227±37%
grug-35b-v2MoEIQ2_S35.1B10.25 GiB0.08 GiB11.14 GiB0.02 GiB227±37%
grug-35bMoEIQ2_S35.1B10.25 GiB0.08 GiB11.14 GiB0.02 GiB227±37%
WorldSim-Opus-3.6-35B-A3BMoEIQ2_S35.1B10.25 GiB0.08 GiB11.14 GiB0.02 GiB227±37%
Qwen3.6-35B-A3B-AnkoMoEIQ2_S35.1B10.25 GiB0.08 GiB11.14 GiB0.02 GiB227±37%
KAT-Coder-V2.5-DevMoEIQ2_S34.7B10.25 GiB0.08 GiB11.14 GiB0.02 GiB227±37%
Ornith-1.0-35BMoEIQ2_S34.7B10.25 GiB0.08 GiB11.14 GiB0.02 GiB227±37%
Nex-N2-miniMoEIQ2_S35.1B10.25 GiB0.08 GiB11.14 GiB0.02 GiB227±37%
Mistral-MOE-4X7B-Dark-MultiVerse-Uncensored-Enhanced32-24BMoEQ3_K_S24.2B9.76 GiB0.53 GiB11.13 GiB0.03 GiB26±37%
Tiger-Gemma-12B-v3Q6_K12.8B9.77 GiB0.51 GiB11.13 GiB0.03 GiB45±12.9%
AfriqueGemma-12BI1-Q6_K12.2B9.77 GiB0.51 GiB11.13 GiB0.03 GiB45±12.9%
Qwen3.5-27B-Engineer-Deckard-GeminiI1-IQ3_XXS27.7B10.00 GiB0.27 GiB11.13 GiB0.03 GiB45±12.9%
Qwen3.5-27B-HERETIC-Polaris-Advanced-Thinking-Alpha-uncensoredI1-IQ3_XXS27.4B10.00 GiB0.27 GiB11.13 GiB0.03 GiB45±12.9%
Qwen3.5-27B-Deckard-PKD-Heretic-Uncensored-ThinkingI1-IQ3_XXS27.4B10.00 GiB0.27 GiB11.13 GiB0.03 GiB45±12.9%
Huihui-Qwen3.5-27B-abliteratedI1-IQ3_XXS27.8B10.00 GiB0.27 GiB11.13 GiB0.03 GiB45±12.9%
Qwen3.5-27B-Unredacted-MAXI1-IQ3_XXS27.4B10.00 GiB0.27 GiB11.13 GiB0.03 GiB45±12.9%
Qwen3.5-27B-hereticI1-IQ3_XXS27.4B10.00 GiB0.27 GiB11.13 GiB0.03 GiB45±12.9%
Qwen3.5-27B-DerestrictedI1-IQ3_XXS27.8B10.00 GiB0.27 GiB11.13 GiB0.03 GiB45±12.9%
Qwen3.5-27B-Claude-4.6-Opus-Reasoning-DistilledI1-IQ3_XXS27.8B10.00 GiB0.27 GiB11.13 GiB0.03 GiB45±12.9%
Grug-12BQ6_K_L12.0B9.76 GiB0.51 GiB11.12 GiB0.04 GiB45±12.9%
gemma-4-12B-it-Esper4Q6_K_L12.0B9.76 GiB0.51 GiB11.12 GiB0.04 GiB45±12.9%
gemma-4-12B-itQ6_K_L12.0B9.76 GiB0.51 GiB11.12 GiB0.04 GiB45±12.9%
Rocinante-XL-16B-v1Q4_K_M16.1B9.37 GiB0.90 GiB11.12 GiB0.04 GiB45±12.9%
EuroLLM-22B-Instruct-2512Q3_K_S22.6B9.35 GiB0.90 GiB11.11 GiB0.05 GiB45±12.9%
solar-pro-preview-instructKV unresolvedQ3_K_S22.1B8.92 GiB1.33 GiB11.11 GiB0.05 GiB45±12.9%
Hermes-4-14BQ5_014.8B9.58 GiB0.66 GiB11.11 GiB0.05 GiB45±12.9%
Nous-Hermes-2-Yi-34BI1-IQ2_XXS34.4B9.23 GiB1.00 GiB11.11 GiB0.05 GiB45±12.9%
Nous-Capybara-limarpv3-34BI1-IQ2_XXS34.4B9.23 GiB1.00 GiB11.11 GiB0.05 GiB45±12.9%
Kimi-Linear-48B-A3B-InstructMoEIQ1_M49.1B10.17 GiB0.13 GiB11.11 GiB0.05 GiB45±12.9%
Qwen3.6-27B-Heretic2-ThinkingI1-Q2_K27.4B9.98 GiB0.27 GiB11.10 GiB0.06 GiB45±12.9%
Qwen3.6-27B-Uncensored-AggressiveI1-Q2_K27.4B9.98 GiB0.27 GiB11.10 GiB0.06 GiB45±12.9%
Qwen-3.5-Opus-GLM-27BI1-Q2_K26.9B9.98 GiB0.27 GiB11.10 GiB0.06 GiB45±12.9%
Qwen3.6-27B-abliteratedI1-Q2_K27.4B9.98 GiB0.27 GiB11.10 GiB0.06 GiB45±12.9%
KoQweopus-3.5-27B-experimentalI1-Q2_K27.8B9.98 GiB0.27 GiB11.10 GiB0.06 GiB45±12.9%
Webcoda-AI-27BI1-Q2_K27.4B9.98 GiB0.27 GiB11.10 GiB0.06 GiB45±12.9%
Qwen3.5-27B-imabari-v2I1-Q2_K27.8B9.98 GiB0.27 GiB11.10 GiB0.06 GiB45±12.9%
Qwen3.6-27B-AEON-Ultimate-Uncensored-BF16Q2_K27.4B9.98 GiB0.27 GiB11.10 GiB0.06 GiB45±12.9%
Qwen3.5-27B-uncensored-heretic-v1I1-Q2_K27.4B9.98 GiB0.27 GiB11.10 GiB0.06 GiB45±12.9%
Carnice-V2-27bI1-Q2_K27.4B9.98 GiB0.27 GiB11.10 GiB0.06 GiB45±12.9%
Qwen3.5-Queen-27BI1-Q2_K27.4B9.98 GiB0.27 GiB11.10 GiB0.06 GiB45±12.9%
GRaPE-2-ProI1-Q2_K27.8B9.98 GiB0.27 GiB11.10 GiB0.06 GiB45±12.9%
Huihui-Qwen3.6-27B-abliteratedQ2_K27.8B9.98 GiB0.27 GiB11.10 GiB0.06 GiB45±12.9%
Qwen3.5-27B-abliteratedQ2_K26.9B9.98 GiB0.27 GiB11.10 GiB0.06 GiB45±12.9%
ThinkingCap-Qwen3.6-27B-hereticQ2_K27.4B9.98 GiB0.27 GiB11.10 GiB0.06 GiB45±12.9%
Qwen-Image-BenchQ2_K27.4B9.98 GiB0.27 GiB11.10 GiB0.06 GiB45±12.9%
Qwen3.6-27B-Claude-Opus-Reasoning-Distill-v2-hereticQ2_K27.4B9.98 GiB0.27 GiB11.10 GiB0.06 GiB45±12.9%
Darwin-28B-REASONI1-Q2_K26.9B9.98 GiB0.27 GiB11.10 GiB0.06 GiB45±12.9%
Huihui-Qwen3.5-27B-Claude-4.6-Opus-abliteratedI1-Q2_K27.8B9.98 GiB0.27 GiB11.10 GiB0.06 GiB45±12.9%
Qwen3.5-27B-WebNovel-Writer-zhI1-Q2_K26.9B9.98 GiB0.27 GiB11.10 GiB0.06 GiB45±12.9%
Qwen3.5-27B_Homebrew-v2I1-Q2_K27.4B9.98 GiB0.27 GiB11.10 GiB0.06 GiB45±12.9%
●From the file◐Predictedwhat these mean

Speed is modeled, not measured: decode is memory-bandwidth bound, so tokens per second is bytes read per token against achievable bandwidth. Mixture-of-experts models carry a wider band because only the routed experts are read each step, and few have been measured publicly.

Questions people ask

What AI models can a Titan V run?
1761 of 2118 indexed open-weight models fit a Titan V at 8,192 context with q8_0 KV cache, the largest being dolphin-2.9.1-mixtral-1x22b at I1-IQ3_M. That covers text, vision-language, image, video and speech models.
How much usable memory does a Titan V actually have?
Its nameplate is 12 GB, but about 11.16 GiB is available to a model once driver and compositor overhead is accounted for.
Is a Titan V fast for local AI?
Its memory bandwidth is 651 GB/s, and that figure — not teraflops — is what governs token generation speed. Capacity decides what you can run; bandwidth decides how fast it runs.