Apple · apple

Apple M2

Apple M2 has 24 GB of unified memory at 102 GB/s — about 16.74 GiB usable after driver and compositor overhead. 1583 of 2118 indexed models fit at 128K context with q8_0 KV. Note only 18 GB of its 24 GB is allocatable to the GPU.

Spec sheet· bandwidth, theoreticalFrom the file· fit from summed bytesPredicted· speed
Memory
24 GB
LPDDR5-6400
Bandwidth
102 GB/s
128-bit bus
Tensor FP16
—
dense
TDP
—
Context4K8K16K32K64K128K
KV cachef16q8_0q4_0quantizing the KV cache is a ~2× lever on the dominant term at long context
vision language 156text 1325video 16embedding 26audio asr 38image 1audio tts 21

What fits at 128K context

largest quantization that fits, per model · 1583 of 2118 indexed
ModelBest quantParamsWeights●KV●Total in memory◐Headroom◐tok/s◐
Qwen3.6-27B-uncensored-heretic-v2-Native-MTP-PreservedQ3_K_M27.4B13.14 GiB4.25 GiB18.00 GiB0.00 GiB5±8.3%
Qwen3.5-27B-uncensored-heretic-v2-Native-MTP-PreservedQ3_K_M27.4B13.14 GiB4.25 GiB18.00 GiB0.00 GiB5±8.3%
dolphin-2.9.2-Phi-3-MediumKV unresolvedIQ2_S14.0B4.11 GiB13.28 GiB18.00 GiB0.00 GiB5±8.3%
Fimbulvetr-11B-v2I1-IQ3_M10.7B4.66 GiB12.75 GiB17.99 GiB0.01 GiB5±8.3%
reka-flash-3.1Q2_K_L20.9B8.60 GiB8.77 GiB17.99 GiB0.01 GiB5±8.3%
reka-flash-3Q2_K_L20.9B8.60 GiB8.77 GiB17.99 GiB0.01 GiB5±8.3%
HomunculusQ4_K_S12.5B6.77 GiB10.63 GiB17.99 GiB0.01 GiB5±8.3%
Wan2.1-VACE-14BQ8_017.3B17.38 GiB0.00 GiB17.97 GiB0.03 GiB5±8.3%
Qwen3.6-35B-A3B-uncensored-heretic-Native-MTP-PreservedMoEQ3_K_M35.1B16.08 GiB1.33 GiB17.96 GiB0.04 GiB16±37%
Qwen3.5-35B-A3B-uncensored-heretic-v2-Native-MTP-PreservedMoEQ3_K_M35.1B16.08 GiB1.33 GiB17.96 GiB0.04 GiB16±37%
Qwen3.6-28BMoEI1-Q4_K_M28.2B16.08 GiB1.33 GiB17.96 GiB0.04 GiB16±37%
Qwen3.5-28BMoEI1-Q4_K_M28.7B16.08 GiB1.33 GiB17.96 GiB0.04 GiB16±37%
Salience-1.5-FlashMoEI1-IQ3_XXS31.1B11.04 GiB6.38 GiB17.95 GiB0.05 GiB6±37%
Huihui-Qwen3-VL-30B-A3B-Instruct-abliteratedMoEI1-IQ3_XXS31.1B11.04 GiB6.38 GiB17.95 GiB0.05 GiB6±37%
Qwen3-30B-A3B-Gemini-Pro-High-Reasoning-2507-ABLITERATED-UNCENSOREDMoEI1-IQ3_XXS30.5B11.04 GiB6.38 GiB17.95 GiB0.05 GiB6±37%
MiroThinker-v1.0-30BMoEI1-IQ3_XXS30.5B11.04 GiB6.38 GiB17.95 GiB0.05 GiB6±37%
Qwen3-30B-A3B-YOYO-V5MoEI1-IQ3_XXS30.5B11.04 GiB6.38 GiB17.95 GiB0.05 GiB6±37%
Qwen3-30B-A3B-Thinking-2507-Claude-4.5-Sonnet-High-Reasoning-DistillMoEI1-IQ3_XXS30.5B11.04 GiB6.38 GiB17.95 GiB0.05 GiB6±37%
Huihui-Qwen3-30B-A3B-Thinking-2507-abliteratedMoEI1-IQ3_XXS30.5B11.04 GiB6.38 GiB17.95 GiB0.05 GiB6±37%
Huihui-Qwen3-30B-A3B-Instruct-2507-abliteratedMoEI1-IQ3_XXS30.5B11.04 GiB6.38 GiB17.95 GiB0.05 GiB6±37%
Qwen3-30B-A3B-abliterated-eroticMoEI1-IQ3_XXS30.5B11.04 GiB6.38 GiB17.95 GiB0.05 GiB6±37%
granite-4.0-h-smallMoEIQ4_XS32.2B16.35 GiB1.06 GiB17.95 GiB0.05 GiB11±37%
medgemma-27b-itI1-IQ3_M28.8B11.69 GiB5.64 GiB17.95 GiB0.05 GiB5±8.3%
gemma-3-27b-it-abliterated-refined-visionI1-IQ3_M27.4B11.69 GiB5.64 GiB17.95 GiB0.05 GiB5±8.3%
Nidum-Gemma-3-27B-it-UncensoredI1-IQ3_M27.4B11.69 GiB5.64 GiB17.95 GiB0.05 GiB5±8.3%
gemma-3-27b-it-abliteratedIQ3_M27.4B11.69 GiB5.64 GiB17.95 GiB0.05 GiB5±8.3%
AtomicGPT-gemma3-27bI1-IQ3_M27.4B11.69 GiB5.64 GiB17.95 GiB0.05 GiB5±8.3%
Unbound-v1.12.0-27BI1-IQ3_M27.4B11.69 GiB5.64 GiB17.95 GiB0.05 GiB5±8.3%
Mira-v1.12-Ties-27BI1-IQ3_M27.4B11.69 GiB5.64 GiB17.95 GiB0.05 GiB5±8.3%
gemma-3-27b-itIQ3_M27.4B11.69 GiB5.64 GiB17.95 GiB0.05 GiB5±8.3%
Medgamma27BI1-IQ3_M27.0B11.69 GiB5.64 GiB17.95 GiB0.05 GiB5±8.3%
Huihui-Qwen3-Coder-30B-A3B-Instruct-abliteratedMoEI1-IQ3_XXS30.5B11.04 GiB6.38 GiB17.95 GiB0.05 GiB6±37%
Qwen3-Coder-30B-A3B-Instruct-RTPurboMoEI1-IQ3_XXS30.5B11.04 GiB6.38 GiB17.95 GiB0.05 GiB6±37%
InternVL3_5-30B-A3BQ4_K_M30.8B17.35 GiB0.00 GiB17.95 GiB0.05 GiB5±8.3%
Le-Chaton-Slim-23BMoEI1-Q3_K_M23.3B10.48 GiB6.91 GiB17.95 GiB0.05 GiB5±37%
Ministral-3-14B-Instruct-2512-BF16-abliteratedI1-Q3_K_L13.9B6.72 GiB10.63 GiB17.95 GiB0.05 GiB5±8.3%
Ministral-3-14B-abliteratedQ3_K_L13.9B6.72 GiB10.63 GiB17.95 GiB0.05 GiB5±8.3%
Ministral-3-14B-Instruct-2512-BF16Q3_K_L13.9B6.72 GiB10.63 GiB17.95 GiB0.05 GiB5±8.3%
Ministral-3-14B-Reasoning-2512-UncensoredI1-Q3_K_L13.9B6.72 GiB10.63 GiB17.95 GiB0.05 GiB5±8.3%
internlm2-math-plus-20bI1-IQ1_M19.9B4.58 GiB12.75 GiB17.94 GiB0.06 GiB5±8.3%
Phi-3-medium-4k-instructI1-IQ2_S14.0B4.04 GiB13.28 GiB17.93 GiB0.07 GiB5±8.3%
Phi-3-medium-128k-instructIQ2_S14.0B4.04 GiB13.28 GiB17.93 GiB0.07 GiB5±8.3%
Qwen3.5-27B-Engineer-Deckard-GeminiI1-Q3_K_L27.7B13.07 GiB4.25 GiB17.93 GiB0.07 GiB5±8.3%
Qwen3.5-27B-HERETIC-Polaris-Advanced-Thinking-Alpha-uncensoredI1-Q3_K_L27.4B13.07 GiB4.25 GiB17.93 GiB0.07 GiB5±8.3%
Qwen3.5-27B-Deckard-PKD-Heretic-Uncensored-ThinkingI1-Q3_K_L27.4B13.07 GiB4.25 GiB17.93 GiB0.07 GiB5±8.3%
Huihui-Qwen3.5-27B-abliteratedI1-Q3_K_L27.8B13.07 GiB4.25 GiB17.93 GiB0.07 GiB5±8.3%
Qwen3.5-27B-Unredacted-MAXI1-Q3_K_L27.4B13.07 GiB4.25 GiB17.93 GiB0.07 GiB5±8.3%
Qwen3.5-27B-hereticI1-Q3_K_L27.4B13.07 GiB4.25 GiB17.93 GiB0.07 GiB5±8.3%
Qwen3.5-27B-DerestrictedI1-Q3_K_L27.8B13.07 GiB4.25 GiB17.93 GiB0.07 GiB5±8.3%
Qwen3.5-27B-Claude-4.6-Opus-Reasoning-DistilledI1-Q3_K_L27.8B13.07 GiB4.25 GiB17.93 GiB0.07 GiB5±8.3%
granite-4.1-8bQ6_K8.8B6.72 GiB10.63 GiB17.93 GiB0.07 GiB5±8.3%
Llama-3.2-8X3B-MOE-Dark-Champion-Instruct-uncensored-abliterated-18.4BMoEQ4_K_S18.4B9.93 GiB7.44 GiB17.93 GiB0.07 GiB5±37%
Nexa-AI-4x4B-InstructMoEI1-Q5_K_S12.1B7.80 GiB9.56 GiB17.92 GiB0.08 GiB4±37%
INTELLECT-1-InstructQ4_K_L10.2B6.16 GiB11.16 GiB17.91 GiB0.09 GiB5±8.3%
Snowpiercer-15B-v4-hereticI1-IQ2_XXS15.0B4.02 GiB13.28 GiB17.90 GiB0.10 GiB5±8.3%
Falcon3-10B-InstructQ5_K_S10.3B6.65 GiB10.63 GiB17.89 GiB0.11 GiB5±8.3%
Gemma-4-12B-StyleTuneQ8_013.0B12.80 GiB4.50 GiB17.89 GiB0.11 GiB5±8.3%
gemma-4-12b-heretic-styletune-headQ8_012.0B12.80 GiB4.50 GiB17.89 GiB0.11 GiB5±8.3%
syrian-gemma-12bQ8_013.0B12.80 GiB4.50 GiB17.89 GiB0.11 GiB5±8.3%
gpt-oss-20b-hereticMoEQ5_K_M20.9B15.74 GiB1.60 GiB17.89 GiB0.11 GiB11±37%
●From the file◐Predictedwhat these mean

Speed is modeled, not measured: decode is memory-bandwidth bound, so tokens per second is bytes read per token against achievable bandwidth. Mixture-of-experts models carry a wider band because only the routed experts are read each step, and few have been measured publicly.

Questions people ask

What AI models can a Apple M2 run?
1583 of 2118 indexed open-weight models fit a Apple M2 at 131,072 context with q8_0 KV cache, the largest being Qwen3.6-27B-uncensored-heretic-v2-Native-MTP-Preserved at Q3_K_M. That covers text, vision-language, image, video and speech models.
How much usable memory does a Apple M2 actually have?
Its nameplate is 24 GB, but about 16.74 GiB is available to a model once driver and compositor overhead is accounted for, and only 18 GB of the pool can be allocated to the GPU at all.
Is a Apple M2 fast for local AI?
Its memory bandwidth is 102 GB/s, and that figure — not teraflops — is what governs token generation speed. Capacity decides what you can run; bandwidth decides how fast it runs.