Apple · apple

Apple M2 Ultra

Apple M2 Ultra has 128 GB of unified memory at 819 GB/s — about 89.28 GiB usable after driver and compositor overhead. 2073 of 2118 indexed models fit at 128K context with q8_0 KV. Note only 96 GB of its 128 GB is allocatable to the GPU.

Spec sheet· bandwidth, theoreticalFrom the file· fit from summed bytesPredicted· speed
Memory
128 GB
LPDDR5-6400
Bandwidth
819 GB/s
1024-bit bus
Tensor FP16
—
dense
TDP
—
Context4K8K16K32K64K128K
KV cachef16q8_0q4_0quantizing the KV cache is a ~2× lever on the dominant term at long context
text 1779vision language 190audio tts 21image 2audio asr 39video 16embedding 26

What fits at 128K context

largest quantization that fits, per model · 2073 of 2118 indexed
ModelBest quantParamsWeights●KV●Total in memory◐Headroom◐tok/s◐
command-r-35b-writer-v2I1-IQ2_XS35.0B10.34 GiB85.00 GiB95.99 GiB0.01 GiB7±8.3%
Llama-4-Scout-17B-16E-InstructMoEKV unresolvedQ6_K109B82.67 GiB12.75 GiB95.99 GiB0.01 GiB16±37%
MiMo-V2-FlashMoEKV unresolvedUD-IQ1_M310B87.36 GiB7.97 GiB95.93 GiB0.07 GiB22±37%
GLM-4.7-REAP-218B-A32BMoEUD-IQ2_M218B70.78 GiB24.44 GiB95.81 GiB0.19 GiB11±37%
Noromaid-20b-v0.1.1I1-Q5_K_S20.0B12.83 GiB82.34 GiB95.76 GiB0.24 GiB7±8.3%
Nethena-20BQ5_020.0B12.83 GiB82.34 GiB95.76 GiB0.24 GiB7±8.3%
Qwen3.5-122B-A10B-hereticMoEI1-Q6_K123B93.42 GiB1.59 GiB95.59 GiB0.41 GiB32±37%
ERNIE-4.5-300B-A47B-PTUD-IQ1_S300B80.54 GiB14.34 GiB95.56 GiB0.44 GiB7±8.3%
Behemoth-X-123B-v2Q4_1123B71.45 GiB23.38 GiB95.53 GiB0.47 GiB7±8.3%
MiniMax-M2.1-REAP-139B-A10BMoEI1-Q4_K_M139B78.40 GiB16.47 GiB95.41 GiB0.59 GiB15±37%
m51Lab-MiniMax-M2.7-REAP-139B-A10BMoEI1-Q4_K_M139B78.40 GiB16.47 GiB95.41 GiB0.59 GiB15±37%
MiMo-V2.5MoEKV unresolvedIQ2_XS311B86.68 GiB7.97 GiB95.25 GiB0.75 GiB22±37%
Mixtral-8x22B-Instruct-v0.1MoEQ4_K_M141B79.71 GiB14.88 GiB95.20 GiB0.80 GiB10±37%
Mixtral-8x22B-v0.1MoEQ4_K_M141B79.71 GiB14.88 GiB95.20 GiB0.80 GiB10±37%
Mixtral-8x22B-v0.1MoEQ4_K_M141B79.71 GiB14.88 GiB95.19 GiB0.81 GiB10±37%
Step-3.7-FlashUD-IQ3_XXS201B68.54 GiB26.05 GiB95.16 GiB0.84 GiB7±8.3%
Laguna-S-2.1MoEUD-Q6_K118B91.19 GiB3.26 GiB95.02 GiB0.98 GiB28±37%
Devstral-2-123B-Instruct-2512Q4_K_L125B70.87 GiB23.38 GiB94.95 GiB1.05 GiB7±8.3%
MiniMax-M2.7MoEUD-IQ3_S229B77.87 GiB16.47 GiB94.88 GiB1.12 GiB16±37%
DeepSeek-Coder-V2-Instruct-0724MoEIQ3_XS236B89.69 GiB4.48 GiB94.76 GiB1.24 GiB26±37%
DeepSeek-V2.5MoEIQ3_XS236B89.69 GiB4.48 GiB94.76 GiB1.24 GiB26±37%
DeepSeek-Coder-V2-InstructMoEIQ3_XS236B89.69 GiB4.48 GiB94.76 GiB1.24 GiB26±37%
MiniMax-M2.1MoEQ2_K_L229B77.72 GiB16.47 GiB94.72 GiB1.28 GiB16±37%
MiniMax-M2.5MoEQ2_K_L229B77.72 GiB16.47 GiB94.72 GiB1.28 GiB16±37%
MiniMax-M2MoEQ2_K_L229B77.72 GiB16.47 GiB94.72 GiB1.28 GiB16±37%
Mistral-Small-4-119B-2603MoEUD-Q6_K119B92.60 GiB1.49 GiB94.68 GiB1.32 GiB33±37%
MiniMax-M2.7-BF16-ultra-uncensored-hereticMoEI1-Q2_K229B77.58 GiB16.47 GiB94.59 GiB1.41 GiB16±37%
Step-3.5-Flash-REAP-121B-A11BI1-Q4_K_M121B67.82 GiB26.05 GiB94.44 GiB1.56 GiB7±8.3%
command-a-plus-05-2026-bf16MoEQ3_K_S219B91.28 GiB2.35 GiB94.19 GiB1.81 GiB26±37%
Qwen2.5-72B-InstructQ8_072.7B72.21 GiB21.25 GiB94.14 GiB1.86 GiB7±8.3%
Qwen3.5-397B-A17BMoEIQ1_M403B91.53 GiB1.99 GiB94.12 GiB1.88 GiB35±37%
GLM-4.6-REAP-268B-A32BMoEUD-IQ1_S269B69.09 GiB24.44 GiB94.12 GiB1.88 GiB11±37%
Kimi-Linear-48B-A3B-InstructMoEBF1649.1B91.54 GiB2.02 GiB94.11 GiB1.89 GiB7±8.3%
HuatuoGPT-o1-72BQ8_072.7B71.96 GiB21.25 GiB93.89 GiB2.11 GiB7±8.3%
Qwen2.5-72B-Instruct-abliteratedQ8_072.7B71.96 GiB21.25 GiB93.89 GiB2.11 GiB7±8.3%
MiroThinker-v1.0-72BQ8_072.7B71.96 GiB21.25 GiB93.89 GiB2.11 GiB7±8.3%
Qwen2.5-Math-72B-InstructQ8_072.7B71.96 GiB21.25 GiB93.89 GiB2.11 GiB7±8.3%
Qwen2.5-72BQ8_072.7B71.96 GiB21.25 GiB93.89 GiB2.11 GiB7±8.3%
Rombo-LLM-V3.0-Qwen-72bQ8_072.7B71.96 GiB21.25 GiB93.89 GiB2.11 GiB7±8.3%
EVA-Qwen2.5-72B-v0.2Q8_072.7B71.96 GiB21.25 GiB93.89 GiB2.11 GiB7±8.3%
Kimi-Dev-72BQ8_072.7B71.96 GiB21.25 GiB93.89 GiB2.11 GiB7±8.3%
Chuluun-Qwen2.5-72B-v0.01Q8_072.7B71.96 GiB21.25 GiB93.89 GiB2.11 GiB7±8.3%
magnum-v4-72bQ8_072.7B71.96 GiB21.25 GiB93.89 GiB2.11 GiB7±8.3%
KAT-Dev-72B-ExpQ8_072.7B71.96 GiB21.25 GiB93.89 GiB2.11 GiB7±8.3%
Homer-v1.0-Qwen2.5-72BQ8_072.7B71.96 GiB21.25 GiB93.89 GiB2.11 GiB7±8.3%
Qwen2.5-VL-72B-InstructQ8_073.4B71.96 GiB21.25 GiB93.89 GiB2.11 GiB7±8.3%
Tower-Plus-72B-ultra-uncensored-hereticQ8_072.7B71.96 GiB21.25 GiB93.89 GiB2.11 GiB7±8.3%
Chronos-Platinum-72BQ8_072.7B71.96 GiB21.25 GiB93.89 GiB2.11 GiB7±8.3%
UI-TARS-72B-DPOQ8_073.4B71.96 GiB21.25 GiB93.89 GiB2.11 GiB7±8.3%
grok-2MoEUD-TQ1_0270B76.17 GiB17.00 GiB93.86 GiB2.14 GiB10±37%
Mistral-Medium-3.5-128BI1-Q4_K_M128B69.75 GiB23.38 GiB93.84 GiB2.16 GiB7±8.3%
XORTRON-NXTXPRTXXLI1-Q4_K_M128B69.75 GiB23.38 GiB93.84 GiB2.16 GiB7±8.3%
Trinity-Large-PreviewMoEIQ2_XXS399B88.58 GiB4.40 GiB93.56 GiB2.44 GiB30±37%
Trinity-Large-TrueBaseMoEIQ2_XXS399B88.58 GiB4.40 GiB93.56 GiB2.44 GiB30±37%
DeepSeek-V4-FlashMoEQ2_K291B92.86 GiB0.03 GiB93.49 GiB2.51 GiB40±37%
Qwen3-VL-235B-A22B-ThinkingMoEQ2_K_L236B79.94 GiB12.48 GiB93.01 GiB2.99 GiB16±37%
Qwen3-VL-235B-A22B-InstructMoEQ2_K_L236B79.94 GiB12.48 GiB93.01 GiB2.99 GiB16±37%
Qwen3-235B-A22BMoEQ2_K_L235B79.94 GiB12.48 GiB93.01 GiB2.99 GiB16±37%
Qwen3-235B-A22B-Instruct-2507MoEQ2_K_L235B79.94 GiB12.48 GiB93.01 GiB2.99 GiB16±37%
Qwen3-235B-A22B-Thinking-2507MoEQ2_K_L235B79.94 GiB12.48 GiB93.01 GiB2.99 GiB16±37%
●From the file◐Predictedwhat these mean

Speed is modeled, not measured: decode is memory-bandwidth bound, so tokens per second is bytes read per token against achievable bandwidth. Mixture-of-experts models carry a wider band because only the routed experts are read each step, and few have been measured publicly.

Questions people ask

What AI models can a Apple M2 Ultra run?
2073 of 2118 indexed open-weight models fit a Apple M2 Ultra at 131,072 context with q8_0 KV cache, the largest being command-r-35b-writer-v2 at I1-IQ2_XS. That covers text, vision-language, image, video and speech models.
How much usable memory does a Apple M2 Ultra actually have?
Its nameplate is 128 GB, but about 89.28 GiB is available to a model once driver and compositor overhead is accounted for, and only 96 GB of the pool can be allocated to the GPU at all.
Is a Apple M2 Ultra fast for local AI?
Its memory bandwidth is 819 GB/s, and that figure — not teraflops — is what governs token generation speed. Capacity decides what you can run; bandwidth decides how fast it runs.