NVIDIA · datacenter

H200 SXM

H200 SXM has 141 GB of VRAM at 4800 GB/s — about 131.13 GiB usable after driver and compositor overhead. 2093 of 2118 indexed models fit at 128K context with q4_0 KV.

Spec sheet· bandwidth, theoreticalFrom the file· fit from summed bytesPredicted· speed
Memory
141 GB
HBM3e
Bandwidth
4800 GB/s
6144-bit bus
Tensor FP16
989 TF
dense
TDP
700 W
Context4K8K16K32K64K128K
KV cachef16q8_0q4_0quantizing the KV cache is a ~2× lever on the dominant term at long context
text 1798vision language 191image 2audio tts 21audio asr 39video 16embedding 26

What fits at 128K context

largest quantization that fits, per model · 2093 of 2118 indexed
ModelBest quantParamsWeights●KV●Total in memory◐Headroom◐tok/s◐
MiMo-V2.5MoEKV unresolvedQ3_K_S311B125.83 GiB4.22 GiB131.10 GiB0.03 GiB96±37%
dots.llm1.instMoEQ5_K_S143B95.08 GiB34.88 GiB130.98 GiB0.15 GiB35±37%
Llama-3_1-Nemotron-51B-InstructQ6_K_L51.5B39.83 GiB90.00 GiB130.97 GiB0.16 GiB21±22%
Qwen3.6-35B-A3B-abliterated-MAXMoEF3235.1B129.13 GiB0.70 GiB130.84 GiB0.29 GiB120±37%
MiniMax-M2.5MoEQ4_K_S229B121.10 GiB8.72 GiB130.81 GiB0.32 GiB77±37%
MiniMax-M2.1MoEI1-Q4_K_S229B121.10 GiB8.72 GiB130.81 GiB0.32 GiB77±37%
MiniMax-M2.7-BF16-ultra-uncensored-hereticMoEQ4_K_S229B121.10 GiB8.72 GiB130.81 GiB0.32 GiB77±37%
MiMo-V2-FlashMoEKV unresolvedI1-IQ3_M310B125.52 GiB4.22 GiB130.79 GiB0.34 GiB96±37%
MiniMax-M2.7MoEQ4_0229B120.93 GiB8.72 GiB130.63 GiB0.50 GiB77±37%
MiniMax-M3MoEUD-IQ2_M427B124.99 GiB4.22 GiB130.22 GiB0.91 GiB97±37%
GLM-4.7-REAP-218B-A32BMoEIQ4_NL218B116.20 GiB12.94 GiB130.17 GiB0.96 GiB53±37%
Hermes-4-405BIQ2_XS406B111.15 GiB17.72 GiB130.15 GiB0.98 GiB21±22%
Hermes-3-Llama-3.1-405BIQ2_XS406B111.15 GiB17.72 GiB130.15 GiB0.98 GiB21±22%
MiniMax-M2MoEIQ4_NL229B120.37 GiB8.72 GiB130.07 GiB1.06 GiB77±37%
step-3.5-flashQ4_1199B115.15 GiB13.79 GiB129.97 GiB1.16 GiB21±22%
Llama-3_3-Nemotron-Super-49B-v1_5Q6_K_L49.9B38.58 GiB90.00 GiB129.72 GiB1.41 GiB21±22%
Valkyrie-49B-v2.1Q6_K_L49.9B38.58 GiB90.00 GiB129.72 GiB1.41 GiB21±22%
Hy3MoEIQ3_XXS299B117.43 GiB11.25 GiB129.72 GiB1.41 GiB66±37%
DeepSeek-V4-FlashMoEUD-IQ4_NL291B128.43 GiB0.02 GiB129.50 GiB1.63 GiB137±37%
Llama-3_3-Nemotron-Super-49B-v1Q6_K49.9B38.11 GiB90.00 GiB129.25 GiB1.88 GiB21±22%
ERNIE-4.5-300B-A47B-PTQ3_K_S300B120.25 GiB7.59 GiB128.97 GiB2.16 GiB21±22%
Step-3.7-FlashUD-Q4_K_M201B113.71 GiB13.79 GiB128.52 GiB2.61 GiB21±22%
DeepSeek-V4-Flash-0731MoEUD-IQ4_NL304B127.28 GiB0.02 GiB128.34 GiB2.79 GiB139±37%
command-a-plus-05-2026-bf16MoEQ4_K_L219B126.06 GiB1.24 GiB128.31 GiB2.82 GiB92±37%
DeepSeek-Coder-V2-Instruct-0724MoEQ4_K_S236B124.68 GiB2.37 GiB128.09 GiB3.04 GiB104±37%
DeepSeek-V2.5MoEQ4_K_S236B124.68 GiB2.37 GiB128.09 GiB3.04 GiB104±37%
DeepSeek-Coder-V2-InstructMoEQ4_K_S236B124.68 GiB2.37 GiB128.09 GiB3.04 GiB104±37%
GLM-4.5MoEUD-IQ2_M358B114.03 GiB12.94 GiB128.01 GiB3.12 GiB61±37%
GLM-4.7MoEUD-IQ2_M358B114.03 GiB12.94 GiB128.01 GiB3.12 GiB61±37%
DeepSeek-V3-0324MoEIQ1_S685B124.38 GiB2.41 GiB127.87 GiB3.26 GiB109±37%
DeepSeek-R1MoEIQ1_S685B124.38 GiB2.41 GiB127.87 GiB3.26 GiB109±37%
GLM-4.6MoEUD-IQ2_M357B113.56 GiB12.94 GiB127.54 GiB3.59 GiB61±37%
Trinity-Large-ThinkingMoEIQ2_M399B123.88 GiB2.33 GiB127.23 GiB3.90 GiB126±37%
granite-34b-code-base-8kF3233.7B125.60 GiB0.00 GiB126.68 GiB4.45 GiB22±22%
Llama-4-Maverick-17B-128E-InstructMoEKV unresolvedUD-IQ1_M402B118.78 GiB6.75 GiB126.56 GiB4.57 GiB101±37%
Trinity-Large-TrueBaseMoEI1-Q2_K_S399B122.88 GiB2.33 GiB126.23 GiB4.90 GiB127±37%
Ornith-1.0-397BMoEIQ2_M397B123.99 GiB1.05 GiB126.09 GiB5.04 GiB135±37%
Qwen3-235B-A22B-abliteratedMoEIQ4_XS235B117.97 GiB6.61 GiB125.61 GiB5.52 GiB73±37%
grok-2MoEIQ3_M270B115.25 GiB9.00 GiB125.39 GiB5.74 GiB36±37%
Qwen3.5-122B-A10BMoEQ8_0125B123.49 GiB0.84 GiB125.36 GiB5.77 GiB123±37%
Qwen3-235B-A22B-Instruct-2507MoEIQ4_XS235B117.24 GiB6.61 GiB124.88 GiB6.25 GiB74±37%
Qwen3-235B-A22B-Thinking-2507MoEIQ4_XS235B117.24 GiB6.61 GiB124.88 GiB6.25 GiB74±37%
Qwen3-235B-A22BMoEIQ4_XS235B116.89 GiB6.61 GiB124.53 GiB6.60 GiB74±37%
Qwen3-VL-235B-A22B-ThinkingMoEIQ4_XS236B116.70 GiB6.61 GiB124.34 GiB6.79 GiB74±37%
Qwen3-VL-235B-A22B-InstructMoEIQ4_XS236B116.70 GiB6.61 GiB124.34 GiB6.79 GiB74±37%
HunyuanImage-2.1Q8_017.5B122.73 GiB0.00 GiB123.78 GiB7.35 GiB22±22%
NVIDIA-Nemotron-3-Super-120B-A12B-BF16MoEQ8_0124B119.65 GiB3.09 GiB123.74 GiB7.39 GiB98±37%
Qwen3.5-122B-A10B-hereticMoEQ8_0123B120.95 GiB0.84 GiB122.82 GiB8.31 GiB125±37%
Laguna-S-2.1MoEQ8_0118B119.91 GiB1.73 GiB122.66 GiB8.47 GiB111±37%
GLM-4.6-REAP-268B-A32BMoEQ3_K_S269B108.47 GiB12.94 GiB122.45 GiB8.68 GiB58±37%
Qwen3.5-REAP-212B-A17BMoEQ4_K_M212B119.56 GiB1.05 GiB121.66 GiB9.47 GiB118±37%
GLM-4.6-Derestricted-v3MoEIQ2_M357B107.14 GiB12.94 GiB121.12 GiB10.01 GiB62±37%
Qwen3.5-397B-A17BMoEIQ2_S403B118.57 GiB1.05 GiB120.68 GiB10.45 GiB140±37%
Trinity-Large-PreviewMoEIQ2_M399B116.50 GiB2.33 GiB119.86 GiB11.27 GiB132±37%
Mistral-Small-4-119B-2603MoEQ8_0119B117.79 GiB0.79 GiB119.61 GiB11.52 GiB129±37%
Solar-Open2-250BMoEQ3_K_M250B111.63 GiB6.75 GiB119.41 GiB11.72 GiB93±37%
Qwen3.5-REAP-262B-A17BMoEQ3_K_M262B116.42 GiB1.05 GiB118.52 GiB12.61 GiB129±37%
gpt-oss-120b-abliteratedMoEQ8_0117B115.76 GiB1.27 GiB118.03 GiB13.10 GiB126±37%
Qwen3-Coder-REAP-363B-A35BMoEUD-IQ1_M363B107.85 GiB8.72 GiB117.60 GiB13.53 GiB69±37%
GLM-4.5-AirMoEQ8_0110B109.39 GiB6.47 GiB116.89 GiB14.24 GiB77±37%
●From the file◐Predictedwhat these mean

Speed is modeled, not measured: decode is memory-bandwidth bound, so tokens per second is bytes read per token against achievable bandwidth. Mixture-of-experts models carry a wider band because only the routed experts are read each step, and few have been measured publicly.

Questions people ask

What AI models can a H200 SXM run?
2093 of 2118 indexed open-weight models fit a H200 SXM at 131,072 context with q4_0 KV cache, the largest being MiMo-V2.5 at Q3_K_S. That covers text, vision-language, image, video and speech models.
How much usable memory does a H200 SXM actually have?
Its nameplate is 141 GB, but about 131.13 GiB is available to a model once driver and compositor overhead is accounted for.
Is a H200 SXM fast for local AI?
Its memory bandwidth is 4800 GB/s, and that figure — not teraflops — is what governs token generation speed. Capacity decides what you can run; bandwidth decides how fast it runs.