Apple · apple

Apple M5

Apple M5 has 24 GB of unified memory at 154 GB/s — about 16.74 GiB usable after driver and compositor overhead. 1927 of 2118 indexed models fit at 16K context with q8_0 KV. Note only 18 GB of its 24 GB is allocatable to the GPU.

Spec sheet· bandwidth, theoreticalFrom the file· fit from summed bytesPredicted· speed
Memory
24 GB
LPDDR5X-9600
Bandwidth
154 GB/s
128-bit bus
Tensor FP16
—
dense
TDP
—
Context4K8K16K32K64K128K
KV cachef16q8_0q4_0quantizing the KV cache is a ~2× lever on the dominant term at long context
text 1652vision language 171audio asr 39video 16image 2audio tts 21embedding 26

What fits at 16K context

largest quantization that fits, per model · 1927 of 2118 indexed
ModelBest quantParamsWeights●KV●Total in memory◐Headroom◐tok/s◐
Qwen3.6-35B-A3B-Fable-5-DistillMoEI1-Q3_K_L36.0B17.28 GiB0.17 GiB18.00 GiB0.00 GiB35±37%
Qwable-v2MoEI1-Q3_K_L36.0B17.28 GiB0.17 GiB18.00 GiB0.00 GiB35±37%
Salience-1.5-ProMoEI1-Q3_K_L36.0B17.28 GiB0.17 GiB18.00 GiB0.00 GiB35±37%
Qwen3.6-35B-A3B-YOYO-V2MoEI1-Q3_K_L36.0B17.28 GiB0.17 GiB18.00 GiB0.00 GiB35±37%
Ornith-1.0-35B-FP8-BLOCK-MTPMoEI1-Q3_K_L35.5B17.28 GiB0.17 GiB18.00 GiB0.00 GiB35±37%
fable-coder-35B-A3BMoEI1-Q3_K_L36.0B17.28 GiB0.17 GiB18.00 GiB0.00 GiB35±37%
Qwen3.6-35B-A3B-AntiLoopMoEI1-Q3_K_L36.0B17.28 GiB0.17 GiB18.00 GiB0.00 GiB35±37%
PINQWEN-3.6-35B-CLEAN-BF16MoEI1-Q3_K_L36.0B17.28 GiB0.17 GiB18.00 GiB0.00 GiB35±37%
UniMath-35B-A3BMoEI1-Q3_K_L36.0B17.28 GiB0.17 GiB18.00 GiB0.00 GiB35±37%
Ornith-1.0-35B-Heretic-MTPMoEI1-Q3_K_L—17.28 GiB0.17 GiB18.00 GiB0.00 GiB35±37%
Fawen-1.0-35BMoEI1-Q3_K_L36.0B17.28 GiB0.17 GiB18.00 GiB0.00 GiB35±37%
CyberStrike-OffSec-35BMoEQ3_K_L35.1B17.28 GiB0.17 GiB18.00 GiB0.00 GiB35±37%
MiniCPM-V-4_5Q8_08.7B16.22 GiB1.20 GiB18.00 GiB0.00 GiB7±8.3%
Qwen3.5-35B-A3B-uncensored-heretic-v2-Native-MTP-PreservedMoEI1-Q3_K_L35.1B17.28 GiB0.17 GiB18.00 GiB0.00 GiB35±37%
Qwen3.6-35B-A3BMoEQ3_K_L36.0B17.28 GiB0.17 GiB18.00 GiB0.00 GiB35±37%
Voxtral-Small-24B-2507Q5_K_L24.3B16.00 GiB1.33 GiB18.00 GiB0.00 GiB7±8.3%
Devstral-Small-2-24B-Instruct-2512Q5_K_L24.0B16.00 GiB1.33 GiB18.00 GiB0.00 GiB7±8.3%
Dolphin3.0-R1-Mistral-24BQ5_K_L23.6B16.00 GiB1.33 GiB17.99 GiB0.01 GiB7±8.3%
Dolphin3.0-Mistral-24BQ5_K_L23.6B16.00 GiB1.33 GiB17.99 GiB0.01 GiB7±8.3%
Cydonia_VistralQ5_K_L23.6B16.00 GiB1.33 GiB17.99 GiB0.01 GiB7±8.3%
Dans-PersonalityEngine-V1.2.0-24bQ5_K_L23.6B16.00 GiB1.33 GiB17.99 GiB0.01 GiB7±8.3%
Dans-PersonalityEngine-V1.3.0-24bQ5_K_L23.6B16.00 GiB1.33 GiB17.99 GiB0.01 GiB7±8.3%
Devstral-Small-2505Q5_K_L23.6B16.00 GiB1.33 GiB17.99 GiB0.01 GiB7±8.3%
Mistral-Small-3.2-24B-Instruct-2506Q5_K_L24.0B16.00 GiB1.33 GiB17.99 GiB0.01 GiB7±8.3%
MS3.2-PaintedFantasy-v3-24BQ5_K_L23.6B16.00 GiB1.33 GiB17.99 GiB0.01 GiB7±8.3%
Precog-24B-v1Q5_K_L—16.00 GiB1.33 GiB17.99 GiB0.01 GiB7±8.3%
Magidonia-24B-v4.3Q5_K_L—16.00 GiB1.33 GiB17.99 GiB0.01 GiB7±8.3%
Magidonia-24B-v4.2.0Q5_K_L23.6B16.00 GiB1.33 GiB17.99 GiB0.01 GiB7±8.3%
MS-2501-DPE-QwQify-v0.1-24BQ5_K_L23.6B16.00 GiB1.33 GiB17.99 GiB0.01 GiB7±8.3%
sarvam-mQ5_K_L23.6B16.00 GiB1.33 GiB17.99 GiB0.01 GiB7±8.3%
Magistral-Small-2506Q5_K_L23.6B16.00 GiB1.33 GiB17.99 GiB0.01 GiB7±8.3%
Cydonia-24B-v4.1Q5_K_L23.6B16.00 GiB1.33 GiB17.99 GiB0.01 GiB7±8.3%
Cydonia-24B-v4Q5_K_L23.6B16.00 GiB1.33 GiB17.99 GiB0.01 GiB7±8.3%
Mistral-Small-3.1-24B-Instruct-2503Q5_K_L24.0B16.00 GiB1.33 GiB17.99 GiB0.01 GiB7±8.3%
Cydonia-24B-v4.3Q5_K_L23.6B16.00 GiB1.33 GiB17.99 GiB0.01 GiB7±8.3%
Cydonia-24B-v4.2.0Q5_K_L23.6B16.00 GiB1.33 GiB17.99 GiB0.01 GiB7±8.3%
Mistral-Small-24B-Instruct-2501-abliteratedQ5_K_L23.6B16.00 GiB1.33 GiB17.99 GiB0.01 GiB7±8.3%
Dolphin-Mistral-24B-Venice-EditionQ5_K_L24.0B16.00 GiB1.33 GiB17.99 GiB0.01 GiB7±8.3%
Mistral-Small-24B-Instruct-2501Q5_K_L23.6B16.00 GiB1.33 GiB17.99 GiB0.01 GiB7±8.3%
Hearthfire-24BQ5_K_L23.6B16.00 GiB1.33 GiB17.99 GiB0.01 GiB7±8.3%
Mistral-Small-24B-ArliAI-RPMax-v1.4Q5_K_L23.6B16.00 GiB1.33 GiB17.99 GiB0.01 GiB7±8.3%
Codex-24B-Small-3.2Q5_K_L23.6B16.00 GiB1.33 GiB17.99 GiB0.01 GiB7±8.3%
Crow-9B-HERETIC-4.6BF169.4B17.14 GiB0.27 GiB17.99 GiB0.01 GiB7±8.3%
Qwen3.5-9B-CoderF169.7B17.14 GiB0.27 GiB17.99 GiB0.01 GiB7±8.3%
Qwythos-9B-Claude-Mythos-5-1M-MTPF169.7B17.14 GiB0.27 GiB17.99 GiB0.01 GiB7±8.3%
Qwen3.5-9B-Fable-5-v1F169.7B17.14 GiB0.27 GiB17.99 GiB0.01 GiB7±8.3%
Huihui-Qwythos-9B-Claude-Mythos-5-1M-abliteratedF169.7B17.14 GiB0.27 GiB17.99 GiB0.01 GiB7±8.3%
PINQWEN-3.5-9B-1M-BF16F169.7B17.14 GiB0.27 GiB17.99 GiB0.01 GiB7±8.3%
Openprose-2-FlashF169.7B17.14 GiB0.27 GiB17.99 GiB0.01 GiB7±8.3%
Qwen3.5-9B-Nikusui-v1F169.7B17.14 GiB0.27 GiB17.99 GiB0.01 GiB7±8.3%
Ornith-1.0-9B-heretic-MTPF169.4B17.14 GiB0.27 GiB17.99 GiB0.01 GiB7±8.3%
Qwen3.5-9BBF169.7B17.14 GiB0.27 GiB17.99 GiB0.01 GiB7±8.3%
Tess-4-9BBF169.7B17.14 GiB0.27 GiB17.99 GiB0.01 GiB7±8.3%
dotwebs-1F169.7B17.14 GiB0.27 GiB17.99 GiB0.01 GiB7±8.3%
liftBF169.7B17.14 GiB0.27 GiB17.99 GiB0.01 GiB7±8.3%
Ornith-1.0-9BBF169.2B17.14 GiB0.27 GiB17.99 GiB0.01 GiB7±8.3%
GLM-4.7-FlashMoEQ4_K31.2B16.99 GiB0.44 GiB17.99 GiB0.01 GiB26±37%
Qwen3.6-35B-A3BMoEUD-IQ4_NL36.0B17.26 GiB0.17 GiB17.98 GiB0.02 GiB35±37%
Qwen3.5-88BMoEI1-IQ1_S87.7B17.20 GiB0.20 GiB17.98 GiB0.02 GiB31±37%
Nemotron-Cascade-2-30B-A3BMoEQ4_031.6B17.02 GiB0.43 GiB17.98 GiB0.02 GiB28±37%
●From the file◐Predictedwhat these mean

Speed is modeled, not measured: decode is memory-bandwidth bound, so tokens per second is bytes read per token against achievable bandwidth. Mixture-of-experts models carry a wider band because only the routed experts are read each step, and few have been measured publicly.

Questions people ask

What AI models can a Apple M5 run?
1927 of 2118 indexed open-weight models fit a Apple M5 at 16,384 context with q8_0 KV cache, the largest being Qwen3.6-35B-A3B-Fable-5-Distill at I1-Q3_K_L. That covers text, vision-language, image, video and speech models.
How much usable memory does a Apple M5 actually have?
Its nameplate is 24 GB, but about 16.74 GiB is available to a model once driver and compositor overhead is accounted for, and only 18 GB of the pool can be allocated to the GPU at all.
Is a Apple M5 fast for local AI?
Its memory bandwidth is 154 GB/s, and that figure — not teraflops — is what governs token generation speed. Capacity decides what you can run; bandwidth decides how fast it runs.