NVIDIA · consumer

Titan V

Titan V has 12 GB of VRAM at 651 GB/s — about 11.16 GiB usable after driver and compositor overhead. 1714 of 2118 indexed models fit at 16K context with q8_0 KV.

Spec sheet· bandwidth, theoreticalFrom the file· fit from summed bytesPredicted· speed
Memory
12 GB
HBM2
Bandwidth
651 GB/s
3072-bit bus
Tensor FP16
119 TF
dense
TDP
250 W
$2999 MSRP
Context4K8K16K32K64K128K
KV cachef16q8_0q4_0quantizing the KV cache is a ~2× lever on the dominant term at long context
text 1468video 14vision language 144audio asr 39audio tts 21image 2embedding 26

What fits at 16K context

largest quantization that fits, per model · 1714 of 2118 indexed
ModelBest quantParamsWeights●KV●Total in memory◐Headroom◐tok/s◐
DeepSeek-V2-Lite-ChatMoEQ5_015.7B10.10 GiB0.25 GiB11.16 GiB0.00 GiB140±37%
gemma-4-19B-A4B-it-INSTRUCT-Heretic-UncensoredMoEI1-IQ4_NL19.0B9.88 GiB0.49 GiB11.16 GiB0.00 GiB45±12.9%
gemma-4-19B-A4B-it-The-DECKARD-Heretic-Uncensored-ThinkingMoEI1-IQ4_NL19.0B9.88 GiB0.49 GiB11.16 GiB0.00 GiB45±12.9%
gemma-4-19b-a4b-it-REAP-hereticMoEI1-IQ4_NL19.0B9.88 GiB0.49 GiB11.16 GiB0.00 GiB45±12.9%
Gemma-4-19BMoEI1-IQ4_NL19.0B9.88 GiB0.49 GiB11.16 GiB0.00 GiB45±12.9%
Le-Chaton-Slim-23BMoEI1-Q3_K_S23.3B9.48 GiB0.86 GiB11.16 GiB0.00 GiB80±37%
Wan2.1-FLF2V-14B-720PQ4_116.4B10.32 GiB0.00 GiB11.16 GiB0.00 GiB45±12.9%
Wan2.1-I2V-14B-480PQ4_116.4B10.32 GiB0.00 GiB11.15 GiB0.01 GiB45±12.9%
Wan2.1-I2V-14B-720PQ4_116.4B10.32 GiB0.00 GiB11.15 GiB0.01 GiB45±12.9%
Phi-4-reasoningQ4_114.7B8.63 GiB1.66 GiB11.15 GiB0.01 GiB45±12.9%
Phi-4-reasoning-plusQ4_114.7B8.63 GiB1.66 GiB11.15 GiB0.01 GiB45±12.9%
phi-4Q4_114.7B8.63 GiB1.66 GiB11.15 GiB0.01 GiB45±12.9%
Ministral-3-14B-Instruct-2512Q5_K_M13.9B8.96 GiB1.33 GiB11.14 GiB0.02 GiB45±12.9%
Ministral-3-14B-Reasoning-2512Q5_K_M13.9B8.96 GiB1.33 GiB11.14 GiB0.02 GiB45±12.9%
Ministral-3-14B-Instruct-2512-BF16-abliteratedI1-Q5_K_M13.9B8.96 GiB1.33 GiB11.14 GiB0.02 GiB45±12.9%
Ministral-3-14B-abliteratedQ5_K_M13.9B8.96 GiB1.33 GiB11.14 GiB0.02 GiB45±12.9%
Ministral-3-14B-Instruct-2512-BF16Q5_K_M13.9B8.96 GiB1.33 GiB11.14 GiB0.02 GiB45±12.9%
Ministral-3-14B-Reasoning-2512-UncensoredI1-Q5_K_M13.9B8.96 GiB1.33 GiB11.14 GiB0.02 GiB45±12.9%
glm-4-9b-chat-1mIQ4_XS9.5B4.98 GiB5.31 GiB11.14 GiB0.02 GiB45±12.9%
diffusiongemma-26B-A4B-it-HERETIC-UncensoredMoEQ2_K25.8B9.86 GiB0.49 GiB11.14 GiB0.02 GiB45±12.9%
diffusiongemma-26B-A4B-itMoEQ2_K25.8B9.86 GiB0.49 GiB11.14 GiB0.02 GiB45±12.9%
Voxtral-Small-24B-2507Q2_K_L24.3B8.89 GiB1.33 GiB11.14 GiB0.02 GiB45±12.9%
Devstral-Small-2-24B-Instruct-2512Q2_K_L24.0B8.89 GiB1.33 GiB11.14 GiB0.02 GiB45±12.9%
Dolphin3.0-R1-Mistral-24BQ2_K_L23.6B8.89 GiB1.33 GiB11.14 GiB0.02 GiB45±12.9%
Dolphin3.0-Mistral-24BQ2_K_L23.6B8.89 GiB1.33 GiB11.14 GiB0.02 GiB45±12.9%
Cydonia_VistralQ2_K_L23.6B8.89 GiB1.33 GiB11.14 GiB0.02 GiB45±12.9%
Dans-PersonalityEngine-V1.2.0-24bQ2_K_L23.6B8.89 GiB1.33 GiB11.14 GiB0.02 GiB45±12.9%
Dans-PersonalityEngine-V1.3.0-24bQ2_K_L23.6B8.89 GiB1.33 GiB11.14 GiB0.02 GiB45±12.9%
Devstral-Small-2505Q2_K_L23.6B8.89 GiB1.33 GiB11.14 GiB0.02 GiB45±12.9%
Mistral-Small-3.2-24B-Instruct-2506Q2_K_L24.0B8.89 GiB1.33 GiB11.14 GiB0.02 GiB45±12.9%
MS3.2-PaintedFantasy-v3-24BQ2_K_L23.6B8.89 GiB1.33 GiB11.14 GiB0.02 GiB45±12.9%
Precog-24B-v1Q2_K_L—8.89 GiB1.33 GiB11.14 GiB0.02 GiB45±12.9%
Magidonia-24B-v4.3Q2_K_L—8.89 GiB1.33 GiB11.14 GiB0.02 GiB45±12.9%
Magidonia-24B-v4.2.0Q2_K_L23.6B8.89 GiB1.33 GiB11.14 GiB0.02 GiB45±12.9%
MS-2501-DPE-QwQify-v0.1-24BQ2_K_L23.6B8.89 GiB1.33 GiB11.14 GiB0.02 GiB45±12.9%
sarvam-mQ2_K_L23.6B8.89 GiB1.33 GiB11.14 GiB0.02 GiB45±12.9%
Magistral-Small-2506Q2_K_L23.6B8.89 GiB1.33 GiB11.14 GiB0.02 GiB45±12.9%
Cydonia-24B-v4.1Q2_K_L23.6B8.89 GiB1.33 GiB11.14 GiB0.02 GiB45±12.9%
Cydonia-24B-v4Q2_K_L23.6B8.89 GiB1.33 GiB11.14 GiB0.02 GiB45±12.9%
Mistral-Small-3.1-24B-Instruct-2503Q2_K_L24.0B8.89 GiB1.33 GiB11.14 GiB0.02 GiB45±12.9%
Cydonia-24B-v4.3Q2_K_L23.6B8.89 GiB1.33 GiB11.14 GiB0.02 GiB45±12.9%
Cydonia-24B-v4.2.0Q2_K_L23.6B8.89 GiB1.33 GiB11.14 GiB0.02 GiB45±12.9%
Mistral-Small-24B-Instruct-2501-abliteratedQ2_K_L23.6B8.89 GiB1.33 GiB11.14 GiB0.02 GiB45±12.9%
Dolphin-Mistral-24B-Venice-EditionQ2_K_L24.0B8.89 GiB1.33 GiB11.14 GiB0.02 GiB45±12.9%
Mistral-Small-24B-Instruct-2501Q2_K_L23.6B8.89 GiB1.33 GiB11.14 GiB0.02 GiB45±12.9%
Hearthfire-24BQ2_K_L23.6B8.89 GiB1.33 GiB11.14 GiB0.02 GiB45±12.9%
Mistral-Small-24B-ArliAI-RPMax-v1.4Q2_K_L23.6B8.89 GiB1.33 GiB11.14 GiB0.02 GiB45±12.9%
Codex-24B-Small-3.2Q2_K_L23.6B8.89 GiB1.33 GiB11.14 GiB0.02 GiB45±12.9%
Qwopus3.6-27B-CoderIQ2_M27.8B9.74 GiB0.53 GiB11.13 GiB0.03 GiB45±12.9%
Goetia-26B-A4B-v1.3-Absolute-Heretic-ARAMoEI1-Q2_K25.8B9.86 GiB0.49 GiB11.13 GiB0.03 GiB45±12.9%
Frank-26B-A4BMoEI1-Q2_K26.5B9.86 GiB0.49 GiB11.13 GiB0.03 GiB45±12.9%
G4-MeroMero-26B-A4B-it-uncensored-hereticMoEI1-Q2_K25.8B9.86 GiB0.49 GiB11.13 GiB0.03 GiB45±12.9%
EVE-26b-XENO-HATMoEI1-Q2_K25.8B9.86 GiB0.49 GiB11.13 GiB0.03 GiB45±12.9%
Gemma-4-26B-A4B-Animus-V14.1-FFT-hereticMoEI1-Q2_K25.8B9.86 GiB0.49 GiB11.13 GiB0.03 GiB45±12.9%
gemma-4-26B-A4B-it-qat-q4_0-unquantized-hereticMoEI1-Q2_K25.8B9.86 GiB0.49 GiB11.13 GiB0.03 GiB45±12.9%
Huihui-gemma-4-26B-A4B-it-qat-q4_0-unquantized-abliteratedMoEI1-Q2_K26.5B9.86 GiB0.49 GiB11.13 GiB0.03 GiB45±12.9%
G4-MeroMero-26B-A4BMoEI1-Q2_K25.8B9.86 GiB0.49 GiB11.13 GiB0.03 GiB45±12.9%
G4-Dark-Soul-26B-A4BMoEI1-Q2_K25.8B9.86 GiB0.49 GiB11.13 GiB0.03 GiB45±12.9%
gemma-4-26B-A4B-it-local-abliterated-sota-internal-t34MoEI1-Q2_K25.8B9.86 GiB0.49 GiB11.13 GiB0.03 GiB45±12.9%
gemma-4-26B-A4B-it-SOMPOA-heresyMoEI1-Q2_K25.8B9.86 GiB0.49 GiB11.13 GiB0.03 GiB45±12.9%
●From the file◐Predictedwhat these mean

Speed is modeled, not measured: decode is memory-bandwidth bound, so tokens per second is bytes read per token against achievable bandwidth. Mixture-of-experts models carry a wider band because only the routed experts are read each step, and few have been measured publicly.

Questions people ask

What AI models can a Titan V run?
1714 of 2118 indexed open-weight models fit a Titan V at 16,384 context with q8_0 KV cache, the largest being DeepSeek-V2-Lite-Chat at Q5_0. That covers text, vision-language, image, video and speech models.
How much usable memory does a Titan V actually have?
Its nameplate is 12 GB, but about 11.16 GiB is available to a model once driver and compositor overhead is accounted for.
Is a Titan V fast for local AI?
Its memory bandwidth is 651 GB/s, and that figure — not teraflops — is what governs token generation speed. Capacity decides what you can run; bandwidth decides how fast it runs.