NVIDIA · datacenter

H100 SXM 80GB

H100 SXM 80GB has 80 GB of VRAM at 3350 GB/s — about 74.40 GiB usable after driver and compositor overhead. 2071 of 2118 indexed models fit at 64K context with q4_0 KV.

Spec sheet· bandwidth, theoreticalFrom the file· fit from summed bytesPredicted· speed
Memory
80 GB
HBM3
Bandwidth
3350 GB/s
5120-bit bus
Tensor FP16
989 TF
dense
TDP
700 W
Context4K8K16K32K64K128K
KV cachef16q8_0q4_0quantizing the KV cache is a ~2× lever on the dominant term at long context
text 1781vision language 186image 2audio asr 39audio tts 21video 16embedding 26

What fits at 64K context

largest quantization that fits, per model · 2071 of 2118 indexed
ModelBest quantParamsWeights●KV●Total in memory◐Headroom◐tok/s◐
MiniMax-M2.5MoEUD-IQ2_XXS229B69.03 GiB4.36 GiB74.38 GiB0.02 GiB99±37%
Qwen3.5-122B-A10BMoEUD-Q4_K_M125B72.89 GiB0.42 GiB74.34 GiB0.06 GiB145±37%
MiniMax-M2.1MoEUD-IQ2_XXS229B68.98 GiB4.36 GiB74.32 GiB0.08 GiB99±37%
Llama-3_3-Nemotron-Super-49B-v1_5Q4_K_M49.9B28.14 GiB45.00 GiB74.28 GiB0.12 GiB26±22%
Valkyrie-49B-v2.1I1-Q4_K_M49.9B28.14 GiB45.00 GiB74.28 GiB0.12 GiB26±22%
Llama-3_3-Nemotron-Super-49B-v1Q4_K_M49.9B28.14 GiB45.00 GiB74.28 GiB0.12 GiB26±22%
OYM-Qimi-122B-A10B-K2.6MoEI1-Q4_1125B72.82 GiB0.42 GiB74.27 GiB0.13 GiB145±37%
MiniMax-M2MoEUD-IQ2_XXS229B68.92 GiB4.36 GiB74.26 GiB0.14 GiB99±37%
c4ai-command-r-plus-08-2024Q5_K_M104B68.57 GiB4.50 GiB74.25 GiB0.15 GiB26±22%
NVIDIA-Nemotron-3-Super-120B-A12B-BF16MoEQ4_1124B71.37 GiB1.55 GiB73.91 GiB0.49 GiB118±37%
DeepSeek-Coder-V2-Instruct-0724MoEIQ2_M236B71.64 GiB1.19 GiB73.86 GiB0.54 GiB127±37%
DeepSeek-V2.5MoEIQ2_M236B71.64 GiB1.19 GiB73.86 GiB0.54 GiB127±37%
DeepSeek-Coder-V2-InstructMoEIQ2_M236B71.64 GiB1.19 GiB73.86 GiB0.54 GiB127±37%
Devstral-2-123B-Instruct-2512Q4_K_S125B66.36 GiB6.19 GiB73.70 GiB0.70 GiB26±22%
Mistral-Medium-3.5-128BI1-Q4_K_S128B66.36 GiB6.19 GiB73.70 GiB0.70 GiB26±22%
XORTRON-NXTXPRTXXLI1-Q4_K_S128B66.36 GiB6.19 GiB73.70 GiB0.70 GiB26±22%
Llama-3_1-Nemotron-51B-InstructQ4_K_S51.5B27.46 GiB45.00 GiB73.60 GiB0.80 GiB26±22%
Llama-4-Scout-17B-16E-InstructMoEKV unresolvedQ5_K_S109B69.16 GiB3.38 GiB73.56 GiB0.84 GiB90±37%
Qwen3.5-REAP-262B-A17BMoEIQ2_XS262B71.81 GiB0.53 GiB73.39 GiB1.01 GiB147±37%
step-3.5-flashQ2_K_L199B65.26 GiB7.04 GiB73.33 GiB1.07 GiB26±22%
GLM-4.5-Air-DerestrictedMoEQ4_K_L110B68.88 GiB3.23 GiB73.14 GiB1.26 GiB91±37%
command-a-plus-05-2026-bf16MoEIQ2_M219B71.32 GiB0.68 GiB73.01 GiB1.39 GiB112±37%
Seed-OSS-36B-InstructBF1636.2B67.35 GiB4.50 GiB72.94 GiB1.46 GiB26±22%
Hermes-4.3-36BBF1636.2B67.35 GiB4.50 GiB72.94 GiB1.46 GiB26±22%
Qwen3.5-122B-A10B-hereticMoEI1-Q4_1123B71.35 GiB0.42 GiB72.80 GiB1.60 GiB148±37%
GLM-4.5-AirMoEQ4_K_M110B68.45 GiB3.23 GiB72.72 GiB1.68 GiB92±37%
Mixtral-8x22B-Instruct-v0.1MoEQ3_K_L141B67.60 GiB3.94 GiB72.60 GiB1.80 GiB45±37%
Mixtral-8x22B-v0.1MoEQ3_K_L141B67.60 GiB3.94 GiB72.59 GiB1.81 GiB45±37%
Mixtral-8x22B-v0.1MoEQ3_K_L141B67.60 GiB3.94 GiB72.59 GiB1.81 GiB45±37%
MiniMax-M2.1-REAP-139B-A10BMoEI1-Q3_K_L139B67.16 GiB4.36 GiB72.51 GiB1.89 GiB89±37%
m51Lab-MiniMax-M2.7-REAP-139B-A10BMoEI1-Q3_K_L139B67.16 GiB4.36 GiB72.51 GiB1.89 GiB89±37%
dots.llm1.instMoEIQ2_M143B53.68 GiB17.44 GiB72.15 GiB2.25 GiB47±37%
Behemoth-X-123B-v2Q4_K_S123B64.79 GiB6.19 GiB72.13 GiB2.27 GiB27±22%
Mistral-Large-Instruct-2411Q4_K_S123B64.79 GiB6.19 GiB72.13 GiB2.27 GiB27±22%
Qwopus3.5-122B-A10B-Kimi-K2.6-destill-healed-abliteratedMoEQ4_K_M123B70.63 GiB0.42 GiB72.08 GiB2.32 GiB149±37%
Step-3.5-Flash-REAP-121B-A11BI1-Q4_K_S121B63.83 GiB7.04 GiB71.90 GiB2.50 GiB27±22%
Step-3.7-FlashIQ2_M201B63.68 GiB7.04 GiB71.75 GiB2.65 GiB27±22%
MiMo-V2-FlashMoEKV unresolvedIQ2_XXS310B68.47 GiB2.11 GiB71.63 GiB2.77 GiB124±37%
CalmeRys-78B-Orpo-v0.1Q6_K78.0B64.27 GiB6.05 GiB71.45 GiB2.95 GiB27±22%
calme-2.3-rys-78bQ6_K78.0B64.27 GiB6.05 GiB71.45 GiB2.95 GiB27±22%
GLM-4.6VMoEQ4_K_L108B66.89 GiB3.23 GiB71.15 GiB3.25 GiB93±37%
Ornith-1.0-35B-AEON-Ultimate-Uncensored-BF16MoEQ8_035.1B69.57 GiB0.35 GiB70.93 GiB3.47 GiB152±37%
Laguna-S-2.1MoEQ4_1118B68.96 GiB0.88 GiB70.86 GiB3.54 GiB134±37%
MiniMax-M2.7MoEUD-IQ2_M229B65.32 GiB4.36 GiB70.67 GiB3.73 GiB102±37%
HarmonicHarlequin_v5-20BQ8_033.3B32.97 GiB36.56 GiB70.58 GiB3.82 GiB27±22%
Qwen2.5-Coder-32B-InstructQ8_032.8B64.86 GiB4.50 GiB70.46 GiB3.94 GiB27±22%
MiMo-V2.5MoEKV unresolvedIQ1_M311B67.01 GiB2.11 GiB70.17 GiB4.23 GiB126±37%
GLM-4.7-REAP-218B-A32BMoEUD-IQ1_M218B62.63 GiB6.47 GiB70.14 GiB4.26 GiB70±37%
Mistral-Small-4-119B-2603MoEUD-Q4_K_M119B68.70 GiB0.40 GiB70.12 GiB4.28 GiB153±37%
gpt-oss-120b-Uncensored-xCloudMoEI1-Q4_1117B68.42 GiB0.64 GiB70.05 GiB4.35 GiB149±37%
gpt-oss-120b-abliteratedMoEI1-Q4_1117B68.42 GiB0.64 GiB70.05 GiB4.35 GiB149±37%
HunyuanImage-2.1Q6_K17.5B68.97 GiB0.00 GiB70.02 GiB4.38 GiB28±22%
GLM-4.5VMoEI1-Q4_K_M108B65.61 GiB3.23 GiB69.87 GiB4.53 GiB94±37%
Qwen3-235B-A22B-abliteratedMoEI1-IQ2_S235B65.40 GiB3.30 GiB69.74 GiB4.66 GiB94±37%
grok-2MoEIQ2_XXS270B63.81 GiB4.50 GiB69.45 GiB4.95 GiB46±37%
Qwen3-VL-235B-A22B-ThinkingMoEUD-IQ1_M236B64.90 GiB3.30 GiB69.24 GiB5.16 GiB94±37%
Qwen3-VL-235B-A22B-InstructMoEUD-IQ1_M236B64.83 GiB3.30 GiB69.17 GiB5.23 GiB95±37%
gpt-oss-20b-hereticMoEIQ4_NL20.9B67.58 GiB0.43 GiB68.99 GiB5.41 GiB84±37%
MiniMax-M2.7-BF16-ultra-uncensored-hereticMoEI1-IQ2_S229B63.36 GiB4.36 GiB68.71 GiB5.69 GiB103±37%
Qwen3.5-88BMoEI1-Q6_K87.7B67.08 GiB0.42 GiB68.53 GiB5.87 GiB139±37%
●From the file◐Predictedwhat these mean

Speed is modeled, not measured: decode is memory-bandwidth bound, so tokens per second is bytes read per token against achievable bandwidth. Mixture-of-experts models carry a wider band because only the routed experts are read each step, and few have been measured publicly.

Questions people ask

What AI models can a H100 SXM 80GB run?
2071 of 2118 indexed open-weight models fit a H100 SXM 80GB at 65,536 context with q4_0 KV cache, the largest being MiniMax-M2.5 at UD-IQ2_XXS. That covers text, vision-language, image, video and speech models.
How much usable memory does a H100 SXM 80GB actually have?
Its nameplate is 80 GB, but about 74.40 GiB is available to a model once driver and compositor overhead is accounted for.
Is a H100 SXM 80GB fast for local AI?
Its memory bandwidth is 3350 GB/s, and that figure — not teraflops — is what governs token generation speed. Capacity decides what you can run; bandwidth decides how fast it runs.