Apple · apple

Apple M4

Apple M4 has 12 GB of unified memory at 120 GB/s — about 8.37 GiB usable after driver and compositor overhead. 1430 of 2118 indexed models fit at 16K context with f16 KV. Note only 9 GB of its 12 GB is allocatable to the GPU.

Spec sheet· bandwidth, theoreticalFrom the file· fit from summed bytesPredicted· speed
Memory
12 GB
LPDDR5X-7500
Bandwidth
120 GB/s
128-bit bus
Tensor FP16
—
dense
TDP
—
Context4K8K16K32K64K128K
KV cachef16q8_0q4_0quantizing the KV cache is a ~2× lever on the dominant term at long context
text 1222vision language 110video 12embedding 26audio asr 38audio tts 21image 1

What fits at 16K context

largest quantization that fits, per model · 1430 of 2118 indexed
ModelBest quantParamsWeights●KV●Total in memory◐Headroom◐tok/s◐
granite-8b-code-instruct-4kI1-Q6_K8.1B6.16 GiB2.25 GiB9.00 GiB0.00 GiB11±8.3%
granite-8b-code-base-4kI1-Q6_K8.1B6.16 GiB2.25 GiB9.00 GiB0.00 GiB11±8.3%
medgemma-27b-itUD-IQ1_M28.8B6.51 GiB1.86 GiB9.00 GiB0.00 GiB12±8.3%
gemma-3-27b-itUD-IQ1_M27.4B6.51 GiB1.86 GiB9.00 GiB0.00 GiB12±8.3%
medgemma-27b-text-itUD-IQ1_M27.0B6.51 GiB1.86 GiB9.00 GiB0.00 GiB12±8.3%
Wan2.1-T2V-14BQ4_014.3B8.41 GiB0.00 GiB9.00 GiB0.00 GiB11±8.3%
Fimbulvetr-11B-v2I1-Q3_K_L10.7B5.41 GiB3.00 GiB8.99 GiB0.01 GiB11±8.3%
Olmo-3-7B-InstructQ5_K_S7.3B4.73 GiB3.69 GiB8.99 GiB0.01 GiB11±8.3%
Olmo-3-7B-ThinkI1-Q5_K_S7.3B4.73 GiB3.69 GiB8.99 GiB0.01 GiB11±8.3%
LFM2-8B-A1BMoEQ8_08.3B8.26 GiB0.19 GiB8.99 GiB0.01 GiB29±37%
Marco-Nano-InstructMoEI1-Q6_K8.0B6.71 GiB1.75 GiB8.99 GiB0.01 GiB22±37%
Apriel-1.6-15b-ThinkerI1-IQ3_XXS14.9B5.39 GiB3.00 GiB8.98 GiB0.02 GiB11±8.3%
InternVL3_5-14BQ4_K_M15.1B8.38 GiB0.00 GiB8.98 GiB0.02 GiB11±8.3%
Nexa-AI-4x4B-InstructMoEIQ4_XS12.1B6.17 GiB2.25 GiB8.98 GiB0.02 GiB10±37%
Mistral-NeMo-Minitron-8B-InstructQ5_K_L8.4B5.90 GiB2.50 GiB8.98 GiB0.02 GiB11±8.3%
Assistant_Pepe_8BQ6_K_L—6.39 GiB2.00 GiB8.98 GiB0.02 GiB11±8.3%
Ling-liteMoEIQ3_M16.8B7.56 GiB0.88 GiB8.98 GiB0.02 GiB25±37%
HunyuanVideo-1.5Q8_08.3B8.38 GiB0.00 GiB8.97 GiB0.03 GiB12±8.3%
Falcon3-10B-InstructQ4_K_M10.3B5.86 GiB2.50 GiB8.97 GiB0.03 GiB12±8.3%
EVA-abliterated-TIES-Qwen2.5-14BI1-Q2_K14.8B5.37 GiB3.00 GiB8.97 GiB0.03 GiB12±8.3%
Neuron-V1-14B-InstructI1-Q2_K14.8B5.37 GiB3.00 GiB8.97 GiB0.03 GiB12±8.3%
Ektome-Qwen2.5-Coder-14B-Instruct-PristinelyUncensoredI1-Q2_K14.8B5.37 GiB3.00 GiB8.97 GiB0.03 GiB12±8.3%
Qwen2.5-14B-Instruct-1M-abliteratedI1-Q2_K14.8B5.37 GiB3.00 GiB8.97 GiB0.03 GiB12±8.3%
DeepCoder-14B-PreviewQ2_K14.8B5.37 GiB3.00 GiB8.97 GiB0.03 GiB12±8.3%
Deepseeker-Kunou-Qwen2.5-14bI1-Q2_K14.8B5.37 GiB3.00 GiB8.97 GiB0.03 GiB12±8.3%
SuperNova-MediusQ2_K14.8B5.37 GiB3.00 GiB8.97 GiB0.03 GiB12±8.3%
14B-Qwen2.5-Kunou-v1I1-Q2_K14.8B5.37 GiB3.00 GiB8.97 GiB0.03 GiB12±8.3%
Sugoi-14B-Ultra-HFI1-Q2_K14.8B5.37 GiB3.00 GiB8.97 GiB0.03 GiB12±8.3%
Qwen2.5-14B-Instruct-abliterated-v2Q2_K14.8B5.37 GiB3.00 GiB8.97 GiB0.03 GiB12±8.3%
Qwen2.5-14B-Instruct-UncensoredQ2_K14.8B5.37 GiB3.00 GiB8.97 GiB0.03 GiB12±8.3%
Qwen2.5-Coder-14B-Instruct-abliteratedQ2_K14.8B5.37 GiB3.00 GiB8.97 GiB0.03 GiB12±8.3%
OpenCodeReasoning-Nemotron-14BQ2_K14.8B5.37 GiB3.00 GiB8.97 GiB0.03 GiB12±8.3%
DeepSeek-R1-Distill-Qwen-14B-abliterated-v2I1-Q2_K14.8B5.37 GiB3.00 GiB8.97 GiB0.03 GiB12±8.3%
C1-TachuI1-Q2_K14.8B5.37 GiB3.00 GiB8.97 GiB0.03 GiB12±8.3%
DeepSeek-R1-Distill-Qwen-14B-abliteratedI1-Q2_K14.8B5.37 GiB3.00 GiB8.97 GiB0.03 GiB12±8.3%
0x-liteQ2_K14.8B5.37 GiB3.00 GiB8.97 GiB0.03 GiB12±8.3%
Qwen2.5-Coder-14B-InstructQ2_K14.8B5.37 GiB3.00 GiB8.97 GiB0.03 GiB12±8.3%
Tessera-4I1-Q2_K14.8B5.37 GiB3.00 GiB8.97 GiB0.03 GiB12±8.3%
AceReason-Nemotron-14BQ2_K14.8B5.37 GiB3.00 GiB8.97 GiB0.03 GiB12±8.3%
Qwen2.5-14B-InstructQ2_K14.8B5.37 GiB3.00 GiB8.97 GiB0.03 GiB12±8.3%
FinetunedQwen14BQ2_K14.8B5.37 GiB3.00 GiB8.97 GiB0.03 GiB12±8.3%
Tessera-4.1I1-Q2_K14.8B5.37 GiB3.00 GiB8.97 GiB0.03 GiB12±8.3%
Qwen2.5-14B-Instruct-1MQ2_K14.8B5.37 GiB3.00 GiB8.97 GiB0.03 GiB12±8.3%
Qwen2.5-Coder-14BQ2_K14.8B5.37 GiB3.00 GiB8.97 GiB0.03 GiB12±8.3%
DeepSeek-R1-Distill-Qwen-14BQ2_K14.8B5.37 GiB3.00 GiB8.97 GiB0.03 GiB12±8.3%
UwU-14B-Math-v0.2I1-Q2_K14.8B5.37 GiB3.00 GiB8.97 GiB0.03 GiB12±8.3%
EVA-Qwen2.5-14B-v0.2I1-Q2_K14.8B5.37 GiB3.00 GiB8.97 GiB0.03 GiB12±8.3%
EVA-Qwen2.5-14B-v0.0I1-Q2_K14.8B5.37 GiB3.00 GiB8.97 GiB0.03 GiB12±8.3%
EVA-Qwen2.5-14B-v0.1I1-Q2_K14.8B5.37 GiB3.00 GiB8.97 GiB0.03 GiB12±8.3%
oxy-1-smallQ2_K14.8B5.37 GiB3.00 GiB8.97 GiB0.03 GiB12±8.3%
Impish_QWEN_14B-1MI1-Q2_K14.8B5.37 GiB3.00 GiB8.97 GiB0.03 GiB12±8.3%
Llama-3.1-Tulu-3-8BQ6_K_L8.0B6.38 GiB2.00 GiB8.97 GiB0.03 GiB11±8.3%
Llama-3-Groq-8B-Tool-UseQ6_K_L8.0B6.38 GiB2.00 GiB8.97 GiB0.03 GiB11±8.3%
Dolphin3.0-Llama3.1-8BQ6_K_L8.0B6.38 GiB2.00 GiB8.97 GiB0.03 GiB11±8.3%
dolphin-2.9.4-llama3.1-8bQ6_K_L8.0B6.38 GiB2.00 GiB8.97 GiB0.03 GiB11±8.3%
dolphin-2.9-llama3-8bQ6_K_L8.0B6.38 GiB2.00 GiB8.97 GiB0.03 GiB11±8.3%
LLAMA-3_8B_Unaligned_BETAQ6_K_L8.0B6.38 GiB2.00 GiB8.97 GiB0.03 GiB11±8.3%
Hypnos-i1-8BQ6_K_L8.0B6.38 GiB2.00 GiB8.97 GiB0.03 GiB11±8.3%
Llama-3.3-8B-Instruct-128K_AbliteratedQ6_K_L8.0B6.38 GiB2.00 GiB8.97 GiB0.03 GiB11±8.3%
Llama-3.1-8B-InstructQ6_K_L8.0B6.38 GiB2.00 GiB8.97 GiB0.03 GiB11±8.3%
●From the file◐Predictedwhat these mean

Speed is modeled, not measured: decode is memory-bandwidth bound, so tokens per second is bytes read per token against achievable bandwidth. Mixture-of-experts models carry a wider band because only the routed experts are read each step, and few have been measured publicly.

Questions people ask

What AI models can a Apple M4 run?
1430 of 2118 indexed open-weight models fit a Apple M4 at 16,384 context with f16 KV cache, the largest being granite-8b-code-instruct-4k at I1-Q6_K. That covers text, vision-language, image, video and speech models.
How much usable memory does a Apple M4 actually have?
Its nameplate is 12 GB, but about 8.37 GiB is available to a model once driver and compositor overhead is accounted for, and only 9 GB of the pool can be allocated to the GPU at all.
Is a Apple M4 fast for local AI?
Its memory bandwidth is 120 GB/s, and that figure — not teraflops — is what governs token generation speed. Capacity decides what you can run; bandwidth decides how fast it runs.