Apple · apple

Apple M2 Pro

Apple M2 Pro has 32 GB of unified memory at 205 GB/s — about 22.32 GiB usable after driver and compositor overhead. 1968 of 2118 indexed models fit at 16K context with q8_0 KV. Note only 24 GB of its 32 GB is allocatable to the GPU.

Spec sheet· bandwidth, theoreticalFrom the file· fit from summed bytesPredicted· speed
Memory
32 GB
LPDDR5-6400
Bandwidth
205 GB/s
256-bit bus
Tensor FP16
—
dense
TDP
—
Context4K8K16K32K64K128K
KV cachef16q8_0q4_0quantizing the KV cache is a ~2× lever on the dominant term at long context
vision language 176text 1688audio tts 21image 2video 16audio asr 39embedding 26

What fits at 16K context

largest quantization that fits, per model · 1968 of 2118 indexed
ModelBest quantParamsWeights●KV●Total in memory◐Headroom◐tok/s◐
Qwen3.6-40B-Claude-4.6-Opus-Deckard-Heretic-Uncensored-ThinkingQ4_K_M39.5B22.59 GiB0.80 GiB24.00 GiB0.00 GiB7±8.3%
Wizard-Vicuna-30B-UncensoredI1-IQ2_M32.5B10.43 GiB12.95 GiB24.00 GiB0.00 GiB7±8.3%
archangel_sft-kto_llama30bI1-IQ2_M32.5B10.43 GiB12.95 GiB24.00 GiB0.00 GiB7±8.3%
GLM-4.7-FlashMoEQ6_K31.2B23.00 GiB0.44 GiB24.00 GiB0.00 GiB27±37%
Nemotron-Cascade-2-30B-A3BMoEQ4_K_M31.6B23.03 GiB0.43 GiB23.99 GiB0.01 GiB29±37%
InternVL3_5-30B-A3BQ6_K30.8B23.38 GiB0.00 GiB23.98 GiB0.02 GiB7±8.3%
Seed-OSS-36B-Instruct-biprojected-norm-preserving-abliteratedI1-Q4_136.2B21.20 GiB2.13 GiB23.97 GiB0.03 GiB7±8.3%
Seed-OSS-36B-InstructQ4_136.2B21.20 GiB2.13 GiB23.97 GiB0.03 GiB7±8.3%
Hermes-4.3-36B-hereticI1-Q4_136.2B21.20 GiB2.13 GiB23.97 GiB0.03 GiB7±8.3%
Hermes-4.3-36BQ4_136.2B21.20 GiB2.13 GiB23.97 GiB0.03 GiB7±8.3%
OmniAtlas-Qwen3-30B-A3BI1-Q6_K31.7B23.37 GiB0.00 GiB23.96 GiB0.04 GiB7±8.3%
Qwen3-Omni-30B-A3B-CaptionerI1-Q6_K31.7B23.37 GiB0.00 GiB23.96 GiB0.04 GiB7±8.3%
Gemma4-Gutenberg-31BQ5_K_L31.3B21.37 GiB1.95 GiB23.96 GiB0.04 GiB7±8.3%
gemma-4-31B-itQ5_K_L31.3B21.37 GiB1.95 GiB23.96 GiB0.04 GiB7±8.3%
Gemma4-Gutenberg-31B-HereticQ5_K_L31.3B21.37 GiB1.95 GiB23.96 GiB0.04 GiB7±8.3%
Equinox-31BQ5_K_L31.3B21.37 GiB1.95 GiB23.96 GiB0.04 GiB7±8.3%
gemma-4-31B-it-SDFT-Heretic-RPQ5_K_L30.7B21.37 GiB1.95 GiB23.96 GiB0.04 GiB7±8.3%
Qwen-AgentWorld-35B-A3BMoEUD-Q5_K_S34.7B23.23 GiB0.17 GiB23.95 GiB0.05 GiB35±37%
Ornith-1.0-35BMoEUD-Q5_K_S34.7B23.23 GiB0.17 GiB23.95 GiB0.05 GiB35±37%
Nemotron-Labs-Audex-30B-A3BQ4_K_L32.0B23.35 GiB0.00 GiB23.95 GiB0.05 GiB7±8.3%
Qwen3-Next-80B-A3B-ThinkingMoEUD-IQ1_M81.3B22.61 GiB0.80 GiB23.95 GiB0.05 GiB32±37%
c4ai-command-r-08-2024Q5_K_L32.3B21.94 GiB1.33 GiB23.93 GiB0.07 GiB7±8.3%
Qwen3-Coder-Next-REAMMoEI1-IQ3_XS60.3B23.19 GiB0.20 GiB23.93 GiB0.07 GiB36±37%
Qwen2.5-Coder-32BQ5_032.8B21.15 GiB2.13 GiB23.92 GiB0.08 GiB7±8.3%
GLM-4.7-Flash-DerestrictedMoEI1-Q6_K31.2B22.92 GiB0.44 GiB23.92 GiB0.08 GiB27±37%
Huihui-GLM-4.7-Flash-abliteratedMoEI1-Q6_K31.2B22.92 GiB0.44 GiB23.92 GiB0.08 GiB27±37%
GLM-4.7-Flash-Claude-Opus-4.5-High-Reasoning-DistillMoEQ6_K31.2B22.92 GiB0.44 GiB23.92 GiB0.08 GiB27±37%
Qwen3.5-122B-A10B-hereticMoEI1-IQ1_S123B23.13 GiB0.20 GiB23.91 GiB0.09 GiB35±37%
WizardCoder-Python-34B-V1.0I1-Q5_K_S33.7B21.64 GiB1.59 GiB23.88 GiB0.12 GiB7±8.3%
Phind-CodeLlama-34B-Python-v1I1-Q5_K_S33.7B21.64 GiB1.59 GiB23.88 GiB0.12 GiB7±8.3%
Phind-CodeLlama-34B-v2I1-Q5_K_S33.7B21.64 GiB1.59 GiB23.88 GiB0.12 GiB7±8.3%
CodeLlama-34b-instruct-hfQ5_033.7B21.64 GiB1.59 GiB23.88 GiB0.12 GiB7±8.3%
WizardLM-1.0-Uncensored-CodeLlama-34bQ5_033.7B21.64 GiB1.59 GiB23.88 GiB0.12 GiB7±8.3%
CallerQ5_K_S32.8B21.08 GiB2.13 GiB23.86 GiB0.14 GiB7±8.3%
Dumpling-Qwen2.5-32BQ5_K_S32.8B21.08 GiB2.13 GiB23.86 GiB0.14 GiB7±8.3%
OREAL-32BQ5_K_S32.8B21.08 GiB2.13 GiB23.86 GiB0.14 GiB7±8.3%
Baichuan-M2-32B-abliteratedQ5_K_S32.8B21.08 GiB2.13 GiB23.86 GiB0.14 GiB7±8.3%
QwQ-32B-Preview-abliterated-linear25I1-Q5_K_S32.8B21.08 GiB2.13 GiB23.86 GiB0.14 GiB7±8.3%
openhands-lm-32b-v0.1I1-Q5_K_S32.8B21.08 GiB2.13 GiB23.86 GiB0.14 GiB7±8.3%
Qwen2.5-Coder-32B-abliteratedI1-Q5_K_S32.8B21.08 GiB2.13 GiB23.86 GiB0.14 GiB7±8.3%
INTELLECT-2Q5_K_S32.8B21.08 GiB2.13 GiB23.86 GiB0.14 GiB7±8.3%
LongWriter-Zero-32BQ5_K_S32.8B21.08 GiB2.13 GiB23.86 GiB0.14 GiB7±8.3%
m1-32bI1-Q5_K_S32.8B21.08 GiB2.13 GiB23.86 GiB0.14 GiB7±8.3%
XMainframe-v2-Instruct-32bI1-Q5_K_S32.8B21.08 GiB2.13 GiB23.86 GiB0.14 GiB7±8.3%
Qwen2.5-Coder-32B-Python-SpecialistI1-Q5_K_S32.8B21.08 GiB2.13 GiB23.86 GiB0.14 GiB7±8.3%
Qwen2.5-32b-RP-InkI1-Q5_K_S32.8B21.08 GiB2.13 GiB23.86 GiB0.14 GiB7±8.3%
OpenCodeReasoning-Nemotron-32B-IOIQ5_K_S32.8B21.08 GiB2.13 GiB23.86 GiB0.14 GiB7±8.3%
Qwen2.5-Coder-32B-Instruct-abliteratedQ5_K_S32.8B21.08 GiB2.13 GiB23.86 GiB0.14 GiB7±8.3%
OlympicCoder-32BQ5_K_S32.8B21.08 GiB2.13 GiB23.86 GiB0.14 GiB7±8.3%
OpenCodeReasoning-Nemotron-32BQ5_K_S32.8B21.08 GiB2.13 GiB23.86 GiB0.14 GiB7±8.3%
OpenThinker-32BQ5_K_S32.8B21.08 GiB2.13 GiB23.86 GiB0.14 GiB7±8.3%
QwQ-32B-ArliAI-RpR-v4Q5_K_S32.8B21.08 GiB2.13 GiB23.86 GiB0.14 GiB7±8.3%
Qwen2.5-Coder-32B-InstructQ5_K_S32.8B21.08 GiB2.13 GiB23.86 GiB0.14 GiB7±8.3%
QwQ-32B-abliteratedQ5_K_S32.8B21.08 GiB2.13 GiB23.86 GiB0.14 GiB7±8.3%
DeepSeek-R1-Distill-Qwen-32B-hereticI1-Q5_K_S32.8B21.08 GiB2.13 GiB23.86 GiB0.14 GiB7±8.3%
InnoSpark-HPC-RM-32BI1-Q5_K_S32.8B21.08 GiB2.13 GiB23.86 GiB0.14 GiB7±8.3%
OpenThinker2-32BQ5_K_S32.8B21.08 GiB2.13 GiB23.86 GiB0.14 GiB7±8.3%
Qwen2.5-32B-InstructQ5_K_S32.8B21.08 GiB2.13 GiB23.86 GiB0.14 GiB7±8.3%
Qwen2.5-Coder-32B-Instruct-UncensoredI1-Q5_K_S32.8B21.08 GiB2.13 GiB23.86 GiB0.14 GiB7±8.3%
QwQ-32B-PreviewQ5_K_S32.8B21.08 GiB2.13 GiB23.86 GiB0.14 GiB7±8.3%
●From the file◐Predictedwhat these mean

Speed is modeled, not measured: decode is memory-bandwidth bound, so tokens per second is bytes read per token against achievable bandwidth. Mixture-of-experts models carry a wider band because only the routed experts are read each step, and few have been measured publicly.

Questions people ask

What AI models can a Apple M2 Pro run?
1968 of 2118 indexed open-weight models fit a Apple M2 Pro at 16,384 context with q8_0 KV cache, the largest being Qwen3.6-40B-Claude-4.6-Opus-Deckard-Heretic-Uncensored-Thinking at Q4_K_M. That covers text, vision-language, image, video and speech models.
How much usable memory does a Apple M2 Pro actually have?
Its nameplate is 32 GB, but about 22.32 GiB is available to a model once driver and compositor overhead is accounted for, and only 24 GB of the pool can be allocated to the GPU at all.
Is a Apple M2 Pro fast for local AI?
Its memory bandwidth is 205 GB/s, and that figure — not teraflops — is what governs token generation speed. Capacity decides what you can run; bandwidth decides how fast it runs.