Apple · apple

Apple M5

Apple M5 has 32 GB of unified memory at 154 GB/s — about 22.32 GiB usable after driver and compositor overhead. 1882 of 2118 indexed models fit at 64K context with q8_0 KV. Note only 24 GB of its 32 GB is allocatable to the GPU.

Spec sheet· bandwidth, theoreticalFrom the file· fit from summed bytesPredicted· speed
Memory
32 GB
LPDDR5X-9600
Bandwidth
154 GB/s
128-bit bus
Tensor FP16
—
dense
TDP
—
Context4K8K16K32K64K128K
KV cachef16q8_0q4_0quantizing the KV cache is a ~2× lever on the dominant term at long context
text 1605vision language 173image 2audio tts 21audio asr 39video 16embedding 26

What fits at 64K context

largest quantization that fits, per model · 1882 of 2118 indexed
ModelBest quantParamsWeights●KV●Total in memory◐Headroom◐tok/s◐
MN-GRAND-23.5B-Gutenberg-UNCENSORED-V2-GLM4.7-ThinkingIQ4_NL23.4B12.64 GiB10.76 GiB24.00 GiB0.00 GiB5±8.3%
Devstral-Small-2-24B-Instruct-2512Q6_K24.0B18.02 GiB5.31 GiB24.00 GiB0.00 GiB5±8.3%
Transformed-Journey-24BI1-Q6_K23.6B18.02 GiB5.31 GiB24.00 GiB0.00 GiB5±8.3%
Magistry-24B-v1.1I1-Q6_K23.6B18.02 GiB5.31 GiB24.00 GiB0.00 GiB5±8.3%
Mergedonia-AETHER-24B-v1aI1-Q6_K23.6B18.02 GiB5.31 GiB24.00 GiB0.00 GiB5±8.3%
Mergedonia-AETHER-24B-v1bI1-Q6_K23.6B18.02 GiB5.31 GiB24.00 GiB0.00 GiB5±8.3%
Slimaki-Tavern-24B-v1.3I1-Q6_K23.6B18.02 GiB5.31 GiB24.00 GiB0.00 GiB5±8.3%
Maginum-Cydoms-24BI1-Q6_K23.6B18.02 GiB5.31 GiB24.00 GiB0.00 GiB5±8.3%
Maginum-Cydoms-24B-absolute-heresyI1-Q6_K23.6B18.02 GiB5.31 GiB24.00 GiB0.00 GiB5±8.3%
Dolphin3.0-Mistral-24BQ6_K23.6B18.02 GiB5.31 GiB24.00 GiB0.00 GiB5±8.3%
Dolphin3.0-R1-Mistral-24BQ6_K23.6B18.02 GiB5.31 GiB24.00 GiB0.00 GiB5±8.3%
Cydonia_VistralQ6_K23.6B18.02 GiB5.31 GiB24.00 GiB0.00 GiB5±8.3%
Mistral-Small-3.2-24B-Instruct-2506-ultra-uncensored-hereticI1-Q6_K24.0B18.02 GiB5.31 GiB24.00 GiB0.00 GiB5±8.3%
Huihui-Mistral-Small-3.2-24B-Instruct-2506-abliterated-llamacppfixedI1-Q6_K24.0B18.02 GiB5.31 GiB24.00 GiB0.00 GiB5±8.3%
Dans-PersonalityEngine-V1.2.0-24bI1-Q6_K23.6B18.02 GiB5.31 GiB24.00 GiB0.00 GiB5±8.3%
Mistral-Small-3_2-24B-Instruct-2506-antislop.v2I1-Q6_K24.0B18.02 GiB5.31 GiB24.00 GiB0.00 GiB5±8.3%
Mistral-Small-3.2-24B-Instruct-2506Q6_K24.0B18.02 GiB5.31 GiB24.00 GiB0.00 GiB5±8.3%
Dans-PersonalityEngine-V1.3.0-24bI1-Q6_K23.6B18.02 GiB5.31 GiB24.00 GiB0.00 GiB5±8.3%
Devstral-Small-2507Q6_K23.6B18.02 GiB5.31 GiB24.00 GiB0.00 GiB5±8.3%
Goetia-24B-v1.1I1-Q6_K23.6B18.02 GiB5.31 GiB24.00 GiB0.00 GiB5±8.3%
Devstral-Small-2505Q6_K23.6B18.02 GiB5.31 GiB24.00 GiB0.00 GiB5±8.3%
MS3.2-PaintedFantasy-v3-24BI1-Q6_K23.6B18.02 GiB5.31 GiB24.00 GiB0.00 GiB5±8.3%
RP-Spectrum-24BI1-Q6_K23.6B18.02 GiB5.31 GiB24.00 GiB0.00 GiB5±8.3%
MS3.2-PaintedFantasy-v4.1-24B-ultra-uncensored-heretic-v2I1-Q6_K23.6B18.02 GiB5.31 GiB24.00 GiB0.00 GiB5±8.3%
Magidonia-24B-v4.3-heretic-v1.2I1-Q6_K23.6B18.02 GiB5.31 GiB24.00 GiB0.00 GiB5±8.3%
Magidonia-24B-v4.3-absolute-heresyI1-Q6_K23.6B18.02 GiB5.31 GiB24.00 GiB0.00 GiB5±8.3%
MagiSeek-Pro-V1I1-Q6_K23.6B18.02 GiB5.31 GiB24.00 GiB0.00 GiB5±8.3%
Magistral-Small-2509Q6_K24.0B18.02 GiB5.31 GiB24.00 GiB0.00 GiB5±8.3%
Magistral-Small-2507Q6_K23.6B18.02 GiB5.31 GiB24.00 GiB0.00 GiB5±8.3%
Cogidonia-v2-24BI1-Q6_K23.6B18.02 GiB5.31 GiB24.00 GiB0.00 GiB5±8.3%
Magidonia-24B-v4.3I1-Q6_K—18.02 GiB5.31 GiB24.00 GiB0.00 GiB5±8.3%
Precog-24B-v1I1-Q6_K—18.02 GiB5.31 GiB24.00 GiB0.00 GiB5±8.3%
experiment024bI1-Q6_K23.6B18.02 GiB5.31 GiB24.00 GiB0.00 GiB5±8.3%
Magidonia-24B-v4.2.0Q6_K23.6B18.02 GiB5.31 GiB24.00 GiB0.00 GiB5±8.3%
Berthier-Mistral-Military-24BI1-Q6_K24.0B18.02 GiB5.31 GiB24.00 GiB0.00 GiB5±8.3%
MS-2501-DPE-QwQify-v0.1-24BQ6_K23.6B18.02 GiB5.31 GiB24.00 GiB0.00 GiB5±8.3%
Mistral-Small-3.2-24B-Instruct-2506-llamacppfixedI1-Q6_K24.0B18.02 GiB5.31 GiB24.00 GiB0.00 GiB5±8.3%
Cydonia-24B-v4.3-absolute-heresyI1-Q6_K23.6B18.02 GiB5.31 GiB24.00 GiB0.00 GiB5±8.3%
Cydonia-24B-v4.3-heretic-v2I1-Q6_K23.6B18.02 GiB5.31 GiB24.00 GiB0.00 GiB5±8.3%
Cydonia-24B-v4.3-hereticI1-Q6_K23.6B18.02 GiB5.31 GiB24.00 GiB0.00 GiB5±8.3%
Cydonia-24B-v4.3-heretic-v4I1-Q6_K23.6B18.02 GiB5.31 GiB24.00 GiB0.00 GiB5±8.3%
Cydonia-24B-v4.2.0I1-Q6_K23.6B18.02 GiB5.31 GiB24.00 GiB0.00 GiB5±8.3%
Journeys-End-24BI1-Q6_K23.6B18.02 GiB5.31 GiB24.00 GiB0.00 GiB5±8.3%
sarvam-mQ6_K23.6B18.02 GiB5.31 GiB24.00 GiB0.00 GiB5±8.3%
Dolphin-Mistral-GLM-4.7-Flash-24B-Venice-Edition-Thinking-UncensoredI1-Q6_K23.6B18.02 GiB5.31 GiB24.00 GiB0.00 GiB5±8.3%
WeirdCompound-v1.7-24bI1-Q6_K23.6B18.02 GiB5.31 GiB24.00 GiB0.00 GiB5±8.3%
Magistral-Small-2506Q6_K23.6B18.02 GiB5.31 GiB24.00 GiB0.00 GiB5±8.3%
Cydonia-24B-v4.3I1-Q6_K23.6B18.02 GiB5.31 GiB24.00 GiB0.00 GiB5±8.3%
Cydonia-24B-v4.1Q6_K23.6B18.02 GiB5.31 GiB24.00 GiB0.00 GiB5±8.3%
Cydonia-24B-v4Q6_K23.6B18.02 GiB5.31 GiB24.00 GiB0.00 GiB5±8.3%
Mistral-Small-3.1-24B-Instruct-2503Q6_K24.0B18.02 GiB5.31 GiB24.00 GiB0.00 GiB5±8.3%
Mistral-Small-24B-Instruct-JbliteratedI1-Q6_K23.6B18.02 GiB5.31 GiB24.00 GiB0.00 GiB5±8.3%
Mistral-Small-24B-Instruct-2501-abliteratedI1-Q6_K23.6B18.02 GiB5.31 GiB24.00 GiB0.00 GiB5±8.3%
MS3.2-24B-Magnum-DiamondQ6_K23.6B18.02 GiB5.31 GiB24.00 GiB0.00 GiB5±8.3%
Dolphin-Mistral-24B-Venice-EditionI1-Q6_K24.0B18.02 GiB5.31 GiB24.00 GiB0.00 GiB5±8.3%
WeirdDolphinPersonalityMechanism-Mistral-24BI1-Q6_K23.6B18.02 GiB5.31 GiB24.00 GiB0.00 GiB5±8.3%
grok-oss-Apollyon-24BI1-Q6_K23.6B18.02 GiB5.31 GiB24.00 GiB0.00 GiB5±8.3%
mistral-small-3.1-24b-instruct-2503-hfQ6_K23.6B18.02 GiB5.31 GiB24.00 GiB0.00 GiB5±8.3%
grok-oss-Apollyon-24B-hereticI1-Q6_K23.6B18.02 GiB5.31 GiB24.00 GiB0.00 GiB5±8.3%
Mistral-Small-24B-Instruct-2501Q6_K23.6B18.02 GiB5.31 GiB24.00 GiB0.00 GiB5±8.3%
●From the file◐Predictedwhat these mean

Speed is modeled, not measured: decode is memory-bandwidth bound, so tokens per second is bytes read per token against achievable bandwidth. Mixture-of-experts models carry a wider band because only the routed experts are read each step, and few have been measured publicly.

Questions people ask

What AI models can a Apple M5 run?
1882 of 2118 indexed open-weight models fit a Apple M5 at 65,536 context with q8_0 KV cache, the largest being MN-GRAND-23.5B-Gutenberg-UNCENSORED-V2-GLM4.7-Thinking at IQ4_NL. That covers text, vision-language, image, video and speech models.
How much usable memory does a Apple M5 actually have?
Its nameplate is 32 GB, but about 22.32 GiB is available to a model once driver and compositor overhead is accounted for, and only 24 GB of the pool can be allocated to the GPU at all.
Is a Apple M5 fast for local AI?
Its memory bandwidth is 154 GB/s, and that figure — not teraflops — is what governs token generation speed. Capacity decides what you can run; bandwidth decides how fast it runs.