Apple · apple

Apple M5

Apple M5 has 24 GB of unified memory at 154 GB/s — about 16.74 GiB usable after driver and compositor overhead. 1863 of 2118 indexed models fit at 64K context with q4_0 KV. Note only 18 GB of its 24 GB is allocatable to the GPU.

Spec sheet· bandwidth, theoreticalFrom the file· fit from summed bytesPredicted· speed
Memory
24 GB
LPDDR5X-9600
Bandwidth
154 GB/s
128-bit bus
Tensor FP16
—
dense
TDP
—
Context4K8K16K32K64K128K
KV cachef16q8_0q4_0quantizing the KV cache is a ~2× lever on the dominant term at long context
text 1588video 16vision language 171embedding 26audio tts 21audio asr 39image 2

What fits at 64K context

largest quantization that fits, per model · 1863 of 2118 indexed
ModelBest quantParamsWeights●KV●Total in memory◐Headroom◐tok/s◐
Hunyuan-A13B-InstructMoEIQ1_S80.4B15.18 GiB2.25 GiB17.98 GiB0.02 GiB7±8.3%
Phi-3-mini-4k-instructKV unresolvedQ6_K3.8B10.67 GiB6.75 GiB17.97 GiB0.03 GiB7±8.3%
Wan2.1-VACE-14BQ8_017.3B17.38 GiB0.00 GiB17.97 GiB0.03 GiB7±8.3%
solar-pro-preview-instructKV unresolvedQ4_K_S22.1B11.73 GiB5.63 GiB17.96 GiB0.04 GiB7±8.3%
L3-Dark-Planet-8BQ6_K8.0B15.12 GiB2.25 GiB17.96 GiB0.04 GiB7±8.3%
codegeex4-all-9bQ5_09.4B6.10 GiB11.25 GiB17.95 GiB0.05 GiB7±8.3%
Orca-2-13b-Alpaca-UncensoredI1-IQ2_XXS13.0B3.30 GiB14.06 GiB17.95 GiB0.05 GiB7±8.3%
WizardLM-13B-UncensoredI1-IQ2_XXS13.0B3.30 GiB14.06 GiB17.95 GiB0.05 GiB7±8.3%
WizardCoder-Python-13B-V1.0I1-IQ2_XXS13.0B3.30 GiB14.06 GiB17.95 GiB0.05 GiB7±8.3%
Guanaco-13B-UncensoredI1-IQ2_XXS13.0B3.30 GiB14.06 GiB17.95 GiB0.05 GiB7±8.3%
InternVL3_5-30B-A3BQ4_K_M30.8B17.35 GiB0.00 GiB17.95 GiB0.05 GiB7±8.3%
gemma-2-27b-itIQ4_XS27.2B13.80 GiB3.46 GiB17.94 GiB0.06 GiB7±8.3%
magnum-v4-27bIQ4_XS27.2B13.80 GiB3.46 GiB17.94 GiB0.06 GiB7±8.3%
Qwen3.6-27B-Fable-5-ExperimentalQ4_K_M27.8B16.20 GiB1.13 GiB17.94 GiB0.06 GiB7±8.3%
dolphin-2.9.2-Phi-3-MediumKV unresolvedQ8_014.0B13.82 GiB3.52 GiB17.94 GiB0.06 GiB7±8.3%
Phi-3-medium-128k-instructQ8_014.0B13.82 GiB3.52 GiB17.94 GiB0.06 GiB7±8.3%
Phi-3-medium-4k-instructQ8_014.0B13.82 GiB3.52 GiB17.94 GiB0.06 GiB7±8.3%
medgemma-27b-itQ4_K_L28.8B15.73 GiB1.58 GiB17.94 GiB0.06 GiB7±8.3%
gemma-3-27b-it-abliteratedQ4_K_L27.4B15.73 GiB1.58 GiB17.94 GiB0.06 GiB7±8.3%
gemma-3-27b-itQ4_K_L27.4B15.73 GiB1.58 GiB17.94 GiB0.06 GiB7±8.3%
Muse-Glimmer-30BQ4_K_L29.8B17.05 GiB0.26 GiB17.94 GiB0.06 GiB7±8.3%
Ornith-1.0-35B-uncensored-hereticMoEQ3_K_L35.1B17.02 GiB0.35 GiB17.93 GiB0.07 GiB32±37%
Qwen3-Coder-Next-Opus-4.6-Reasoning-DistilledMoEIQ1_M—16.95 GiB0.42 GiB17.91 GiB0.09 GiB34±37%
CallerIQ3_XS32.8B12.76 GiB4.50 GiB17.91 GiB0.09 GiB7±8.3%
Dumpling-Qwen2.5-32BIQ3_XS32.8B12.76 GiB4.50 GiB17.91 GiB0.09 GiB7±8.3%
OREAL-32BIQ3_XS32.8B12.76 GiB4.50 GiB17.91 GiB0.09 GiB7±8.3%
QwQ-32B-Preview-abliterated-linear25I1-IQ3_XS32.8B12.76 GiB4.50 GiB17.91 GiB0.09 GiB7±8.3%
openhands-lm-32b-v0.1I1-IQ3_XS32.8B12.76 GiB4.50 GiB17.91 GiB0.09 GiB7±8.3%
Qwen2.5-Coder-32B-abliteratedI1-IQ3_XS32.8B12.76 GiB4.50 GiB17.91 GiB0.09 GiB7±8.3%
m1-32bI1-IQ3_XS32.8B12.76 GiB4.50 GiB17.91 GiB0.09 GiB7±8.3%
XMainframe-v2-Instruct-32bI1-IQ3_XS32.8B12.76 GiB4.50 GiB17.91 GiB0.09 GiB7±8.3%
Qwen2.5-Coder-32B-Python-SpecialistI1-IQ3_XS32.8B12.76 GiB4.50 GiB17.91 GiB0.09 GiB7±8.3%
Qwen2.5-32b-RP-InkI1-IQ3_XS32.8B12.76 GiB4.50 GiB17.91 GiB0.09 GiB7±8.3%
LongWriter-Zero-32BIQ3_XS32.8B12.76 GiB4.50 GiB17.91 GiB0.09 GiB7±8.3%
OpenCodeReasoning-Nemotron-32B-IOIIQ3_XS32.8B12.76 GiB4.50 GiB17.91 GiB0.09 GiB7±8.3%
Qwen2.5-Coder-32B-Instruct-abliteratedIQ3_XS32.8B12.76 GiB4.50 GiB17.91 GiB0.09 GiB7±8.3%
OlympicCoder-32BIQ3_XS32.8B12.76 GiB4.50 GiB17.91 GiB0.09 GiB7±8.3%
OpenCodeReasoning-Nemotron-32BIQ3_XS32.8B12.76 GiB4.50 GiB17.91 GiB0.09 GiB7±8.3%
OpenThinker-32BIQ3_XS32.8B12.76 GiB4.50 GiB17.91 GiB0.09 GiB7±8.3%
QwQ-32B-ArliAI-RpR-v4IQ3_XS32.8B12.76 GiB4.50 GiB17.91 GiB0.09 GiB7±8.3%
Qwen2.5-Coder-32B-InstructIQ3_XS32.8B12.76 GiB4.50 GiB17.91 GiB0.09 GiB7±8.3%
Qwen2.5-Coder-32BIQ3_XS32.8B12.76 GiB4.50 GiB17.91 GiB0.09 GiB7±8.3%
QwQ-32B-abliteratedIQ3_XS32.8B12.76 GiB4.50 GiB17.91 GiB0.09 GiB7±8.3%
DeepSeek-R1-Distill-Qwen-32B-hereticI1-IQ3_XS32.8B12.76 GiB4.50 GiB17.91 GiB0.09 GiB7±8.3%
InnoSpark-HPC-RM-32BI1-IQ3_XS32.8B12.76 GiB4.50 GiB17.91 GiB0.09 GiB7±8.3%
OpenThinker2-32BIQ3_XS32.8B12.76 GiB4.50 GiB17.91 GiB0.09 GiB7±8.3%
INTELLECT-2IQ3_XS32.8B12.76 GiB4.50 GiB17.91 GiB0.09 GiB7±8.3%
Qwen2.5-32B-InstructIQ3_XS32.8B12.76 GiB4.50 GiB17.91 GiB0.09 GiB7±8.3%
Qwen2.5-Coder-32B-Instruct-UncensoredI1-IQ3_XS32.8B12.76 GiB4.50 GiB17.91 GiB0.09 GiB7±8.3%
QwQ-32B-PreviewIQ3_XS32.8B12.76 GiB4.50 GiB17.91 GiB0.09 GiB7±8.3%
TinyR1-32B-PreviewIQ3_XS32.8B12.76 GiB4.50 GiB17.91 GiB0.09 GiB7±8.3%
deepseek-r1-qwen-2.5-32B-ablatedIQ3_XS32.8B12.76 GiB4.50 GiB17.91 GiB0.09 GiB7±8.3%
Rombos-LLM-V2.5-Qwen-32bIQ3_XS32.8B12.76 GiB4.50 GiB17.91 GiB0.09 GiB7±8.3%
DeepSeek-R1-Distill-Qwen-32B-abliteratedIQ3_XS32.8B12.76 GiB4.50 GiB17.91 GiB0.09 GiB7±8.3%
Qwen2.5-32B-ArliAI-RPMax-v1.3IQ3_XS32.8B12.76 GiB4.50 GiB17.91 GiB0.09 GiB7±8.3%
DeepSeek-R1-Distill-Qwen-32BIQ3_XS32.8B12.76 GiB4.50 GiB17.91 GiB0.09 GiB7±8.3%
Qwen2.5-VL-32B-InstructIQ3_XS33.5B12.76 GiB4.50 GiB17.91 GiB0.09 GiB7±8.3%
EVA-Qwen2.5-32B-v0.2IQ3_XS32.8B12.76 GiB4.50 GiB17.91 GiB0.09 GiB7±8.3%
EVA-Qwen2.5-32B-v0.1IQ3_XS32.8B12.76 GiB4.50 GiB17.91 GiB0.09 GiB7±8.3%
cogito-v1-preview-qwen-32BI1-IQ3_XS32.8B12.76 GiB4.50 GiB17.91 GiB0.09 GiB7±8.3%
●From the file◐Predictedwhat these mean

Speed is modeled, not measured: decode is memory-bandwidth bound, so tokens per second is bytes read per token against achievable bandwidth. Mixture-of-experts models carry a wider band because only the routed experts are read each step, and few have been measured publicly.

Questions people ask

What AI models can a Apple M5 run?
1863 of 2118 indexed open-weight models fit a Apple M5 at 65,536 context with q4_0 KV cache, the largest being Hunyuan-A13B-Instruct at IQ1_S. That covers text, vision-language, image, video and speech models.
How much usable memory does a Apple M5 actually have?
Its nameplate is 24 GB, but about 16.74 GiB is available to a model once driver and compositor overhead is accounted for, and only 18 GB of the pool can be allocated to the GPU at all.
Is a Apple M5 fast for local AI?
Its memory bandwidth is 154 GB/s, and that figure — not teraflops — is what governs token generation speed. Capacity decides what you can run; bandwidth decides how fast it runs.