Apple · apple

Apple M2

Apple M2 has 24 GB of unified memory at 102 GB/s — about 16.74 GiB usable after driver and compositor overhead. 1943 of 2118 indexed models fit at 16K context with q4_0 KV. Note only 18 GB of its 24 GB is allocatable to the GPU.

Spec sheet· bandwidth, theoreticalFrom the file· fit from summed bytesPredicted· speed
Memory
24 GB
LPDDR5-6400
Bandwidth
102 GB/s
128-bit bus
Tensor FP16
—
dense
TDP
—
Context4K8K16K32K64K128K
KV cachef16q8_0q4_0quantizing the KV cache is a ~2× lever on the dominant term at long context
text 1668video 16vision language 171audio asr 39image 2audio tts 21embedding 26

What fits at 16K context

largest quantization that fits, per model · 1943 of 2118 indexed
ModelBest quantParamsWeights●KV●Total in memory◐Headroom◐tok/s◐
Hunyuan-A13B-InstructMoEIQ1_M80.4B16.87 GiB0.56 GiB17.98 GiB0.02 GiB5±8.3%
Delphi-25B-SimpleRL-MathIQ4_XS25.0B12.65 GiB4.71 GiB17.98 GiB0.02 GiB5±8.3%
c4ai-command-r-08-2024IQ4_XS32.3B16.60 GiB0.70 GiB17.97 GiB0.03 GiB5±8.3%
Wan2.1-VACE-14BQ8_017.3B17.38 GiB0.00 GiB17.97 GiB0.03 GiB5±8.3%
EXAONE-4.5-33BIQ4_XS34.4B16.79 GiB0.52 GiB17.96 GiB0.04 GiB5±8.3%
Gemma-3-27B-MeditronFOI1-Q4_128.8B16.81 GiB0.52 GiB17.96 GiB0.04 GiB5±8.3%
Hy-MT2-30B-A3BMoEQ4_K_M30.1B16.98 GiB0.42 GiB17.95 GiB0.05 GiB18±37%
Qwen3-48B-A4B-Savant-Commander-Distill-12X-Closed-Open-Heretic-UncensoredMoEI1-IQ4_XS33.6B16.76 GiB0.63 GiB17.95 GiB0.05 GiB11±37%
InternVL3_5-30B-A3BQ4_K_M30.8B17.35 GiB0.00 GiB17.95 GiB0.05 GiB5±8.3%
Gemma-4-31B-Isometry-RPI1-IQ4_XS32.7B16.28 GiB1.03 GiB17.95 GiB0.05 GiB5±8.3%
Gemma-4-Dark-Gemistry-31BI1-IQ4_XS32.7B16.28 GiB1.03 GiB17.95 GiB0.05 GiB5±8.3%
Prosopon-31BI1-IQ4_XS32.7B16.28 GiB1.03 GiB17.95 GiB0.05 GiB5±8.3%
Gemma-4-Novelist-Eclipse-31BI1-IQ4_XS32.7B16.28 GiB1.03 GiB17.95 GiB0.05 GiB5±8.3%
Giftige-Blume-31B-v1-StyleSwapI1-IQ4_XS32.7B16.28 GiB1.03 GiB17.95 GiB0.05 GiB5±8.3%
G4-MeroMero-31B-StyleSwapI1-IQ4_XS32.7B16.28 GiB1.03 GiB17.95 GiB0.05 GiB5±8.3%
Gemma-4-31B-StyleTune-heretic-araI1-IQ4_XS32.7B16.28 GiB1.03 GiB17.95 GiB0.05 GiB5±8.3%
Pantheon-Reasoning-31B-1.1I1-IQ4_XS32.7B16.28 GiB1.03 GiB17.95 GiB0.05 GiB5±8.3%
Gemma-4-31B-StyleTuneI1-IQ4_XS32.7B16.28 GiB1.03 GiB17.95 GiB0.05 GiB5±8.3%
Barcenas-StyleTune-31B-FableI1-IQ4_XS32.1B16.28 GiB1.03 GiB17.95 GiB0.05 GiB5±8.3%
G4-MeroMero-26B-A4B-it-uncensored-hereticMoEQ5_K_S25.8B17.14 GiB0.26 GiB17.93 GiB0.07 GiB5±8.3%
Phi-3.5-MoE-instructMoEKV unresolvedQ3_K_S41.9B16.82 GiB0.56 GiB17.93 GiB0.07 GiB13±37%
Aurora-Code-1MoEI1-Q4_K_M34.7B17.28 GiB0.09 GiB17.92 GiB0.08 GiB26±37%
Qwen3.6-35B-A3B-Fable-5-DistillMoEI1-Q3_K_L36.0B17.28 GiB0.09 GiB17.92 GiB0.08 GiB26±37%
Qwable-v2MoEI1-Q3_K_L36.0B17.28 GiB0.09 GiB17.92 GiB0.08 GiB26±37%
Salience-1.5-ProMoEI1-Q3_K_L36.0B17.28 GiB0.09 GiB17.92 GiB0.08 GiB26±37%
Qwen3.6-35B-A3B-YOYO-V2MoEI1-Q3_K_L36.0B17.28 GiB0.09 GiB17.92 GiB0.08 GiB26±37%
Ornith-1.0-35B-FP8-BLOCK-MTPMoEI1-Q3_K_L35.5B17.28 GiB0.09 GiB17.92 GiB0.08 GiB26±37%
fable-coder-35B-A3BMoEI1-Q3_K_L36.0B17.28 GiB0.09 GiB17.92 GiB0.08 GiB26±37%
Qwen3.6-35B-A3B-AntiLoopMoEI1-Q3_K_L36.0B17.28 GiB0.09 GiB17.92 GiB0.08 GiB26±37%
PINQWEN-3.6-35B-CLEAN-BF16MoEI1-Q3_K_L36.0B17.28 GiB0.09 GiB17.92 GiB0.08 GiB26±37%
UniMath-35B-A3BMoEI1-Q3_K_L36.0B17.28 GiB0.09 GiB17.92 GiB0.08 GiB26±37%
Ornith-1.0-35B-Heretic-MTPMoEI1-Q3_K_L—17.28 GiB0.09 GiB17.92 GiB0.08 GiB26±37%
Fawen-1.0-35BMoEI1-Q3_K_L36.0B17.28 GiB0.09 GiB17.92 GiB0.08 GiB26±37%
CyberStrike-OffSec-35BMoEQ3_K_L35.1B17.28 GiB0.09 GiB17.92 GiB0.08 GiB26±37%
Qwen3.5-35B-A3B-uncensored-heretic-v2-Native-MTP-PreservedMoEI1-Q3_K_L35.1B17.28 GiB0.09 GiB17.92 GiB0.08 GiB26±37%
Qwen3.6-35B-A3BMoEQ3_K_L36.0B17.28 GiB0.09 GiB17.92 GiB0.08 GiB26±37%
gemma-4-E2B-it-Uncensored-MAXF325.1B17.33 GiB0.04 GiB17.91 GiB0.09 GiB5±8.3%
OpenBuddy-R1-0528-Distill-Qwen3-32B-Preview0-QATQ3_K_L32.8B16.14 GiB1.13 GiB17.91 GiB0.09 GiB5±8.3%
Qwen3-VL-32B-Instruct-ultra-uncensored-hereticI1-Q3_K_L33.4B16.14 GiB1.13 GiB17.91 GiB0.09 GiB5±8.3%
Huihui-Qwen3-VL-32B-Instruct-abliteratedI1-Q3_K_L33.4B16.14 GiB1.13 GiB17.91 GiB0.09 GiB5±8.3%
KAT-DevQ3_K_L32.8B16.14 GiB1.13 GiB17.91 GiB0.09 GiB5±8.3%
ColorGUI-32BI1-Q3_K_L33.4B16.14 GiB1.13 GiB17.91 GiB0.09 GiB5±8.3%
Qwen3-VL-32B-InstructQ3_K_L33.4B16.14 GiB1.13 GiB17.91 GiB0.09 GiB5±8.3%
Qwen3-32B-UncensoredI1-Q3_K_L32.8B16.14 GiB1.13 GiB17.91 GiB0.09 GiB5±8.3%
Qwen3-32B-abliteratedI1-Q3_K_L32.8B16.14 GiB1.13 GiB17.91 GiB0.09 GiB5±8.3%
DeepSWE-PreviewQ3_K_L32.8B16.14 GiB1.13 GiB17.91 GiB0.09 GiB5±8.3%
AReaL-boba-2-32BI1-Q3_K_L32.8B16.14 GiB1.13 GiB17.91 GiB0.09 GiB5±8.3%
Qwen3-32BQ3_K_L32.8B16.14 GiB1.13 GiB17.91 GiB0.09 GiB5±8.3%
Assistant_Pepe_32BI1-Q3_K_L32.8B16.14 GiB1.13 GiB17.91 GiB0.09 GiB5±8.3%
Qwen3.6-35B-A3BMoEUD-IQ4_NL36.0B17.26 GiB0.09 GiB17.91 GiB0.09 GiB26±37%
Wizard-Vicuna-30B-UncensoredI1-IQ2_M32.5B10.43 GiB6.86 GiB17.90 GiB0.10 GiB5±8.3%
archangel_sft-kto_llama30bI1-IQ2_M32.5B10.43 GiB6.86 GiB17.90 GiB0.10 GiB5±8.3%
mistral-small-3.1-24b-instruct-2503-hfQ5_123.6B16.52 GiB0.70 GiB17.89 GiB0.11 GiB5±8.3%
Qwen3.5-88BMoEI1-IQ1_S87.7B17.20 GiB0.11 GiB17.89 GiB0.11 GiB23±37%
Apertus-70B-Instruct-2509IQ1_M70.6B15.74 GiB1.41 GiB17.88 GiB0.12 GiB5±8.3%
OmniAtlas-Qwen3-30B-A3BI1-Q4_K_M31.7B17.28 GiB0.00 GiB17.88 GiB0.12 GiB5±8.3%
Qwen3-Omni-30B-A3B-InstructQ4_K_M35.3B17.28 GiB0.00 GiB17.88 GiB0.12 GiB5±8.3%
Qwen3-Omni-30B-A3B-CaptionerI1-Q4_K_M31.7B17.28 GiB0.00 GiB17.88 GiB0.12 GiB5±8.3%
Qwen3-Omni-30B-A3B-ThinkingQ4_K_M31.7B17.28 GiB0.00 GiB17.88 GiB0.12 GiB5±8.3%
Crow-9B-HERETIC-4.6BF169.4B17.14 GiB0.14 GiB17.87 GiB0.13 GiB5±8.3%
●From the file◐Predictedwhat these mean

Speed is modeled, not measured: decode is memory-bandwidth bound, so tokens per second is bytes read per token against achievable bandwidth. Mixture-of-experts models carry a wider band because only the routed experts are read each step, and few have been measured publicly.

Questions people ask

What AI models can a Apple M2 run?
1943 of 2118 indexed open-weight models fit a Apple M2 at 16,384 context with q4_0 KV cache, the largest being Hunyuan-A13B-Instruct at IQ1_M. That covers text, vision-language, image, video and speech models.
How much usable memory does a Apple M2 actually have?
Its nameplate is 24 GB, but about 16.74 GiB is available to a model once driver and compositor overhead is accounted for, and only 18 GB of the pool can be allocated to the GPU at all.
Is a Apple M2 fast for local AI?
Its memory bandwidth is 102 GB/s, and that figure — not teraflops — is what governs token generation speed. Capacity decides what you can run; bandwidth decides how fast it runs.