Apple · apple

Apple M4

Apple M4 has 12 GB of unified memory at 120 GB/s — about 8.37 GiB usable after driver and compositor overhead. 1146 of 2118 indexed models fit at 128K context with q4_0 KV. Note only 9 GB of its 12 GB is allocatable to the GPU.

Spec sheet· bandwidth, theoreticalFrom the file· fit from summed bytesPredicted· speed
Memory
12 GB
LPDDR5X-7500
Bandwidth
120 GB/s
128-bit bus
Tensor FP16
—
dense
TDP
—
Context4K8K16K32K64K128K
KV cachef16q8_0q4_0quantizing the KV cache is a ~2× lever on the dominant term at long context
text 950video 12vision language 99embedding 26audio asr 38audio tts 20image 1

What fits at 128K context

largest quantization that fits, per model · 1146 of 2118 indexed
ModelBest quantParamsWeights●KV●Total in memory◐Headroom◐tok/s◐
Qwen3-4B-BaseQ6_K4.0B3.38 GiB5.06 GiB9.00 GiB0.00 GiB11±8.3%
legitus-instruct-v1I1-Q3_K_M8.1B3.88 GiB4.50 GiB9.00 GiB0.00 GiB12±8.3%
Apertus-8B-Instruct-2509I1-Q3_K_M8.1B3.88 GiB4.50 GiB9.00 GiB0.00 GiB12±8.3%
Wan2.1-T2V-14BQ4_014.3B8.41 GiB0.00 GiB9.00 GiB0.00 GiB11±8.3%
SmolLM2-1.7B-InstructQ8_01.7B1.70 GiB6.75 GiB8.99 GiB0.01 GiB11±8.3%
internlm3-8b-instructQ6_K8.8B6.73 GiB1.69 GiB8.99 GiB0.01 GiB11±8.3%
Qwen3.6-35B-A3B-REAM-160-ru-agentMoEIQ2_M23.6B7.73 GiB0.70 GiB8.99 GiB0.01 GiB31±37%
Ministral-3-8B-Instruct-2512-BF16-abliteratedI1-IQ3_S8.9B3.62 GiB4.78 GiB8.99 GiB0.01 GiB11±8.3%
Amaretto-8BI1-IQ3_S8.9B3.62 GiB4.78 GiB8.99 GiB0.01 GiB11±8.3%
rnj-1-instructQ3_K_M8.3B3.89 GiB4.50 GiB8.99 GiB0.01 GiB11±8.3%
Parable-Granite-4.1-8B-Claude-Fable-5I1-IQ2_M8.4B2.78 GiB5.63 GiB8.99 GiB0.01 GiB11±8.3%
INTELLECT-1-InstructI1-IQ1_M10.2B2.48 GiB5.91 GiB8.98 GiB0.02 GiB11±8.3%
InternVL3_5-14BQ4_K_M15.1B8.38 GiB0.00 GiB8.98 GiB0.02 GiB11±8.3%
Tiger-Gemma-12B-v3Q3_K_M12.8B6.00 GiB2.38 GiB8.98 GiB0.02 GiB11±8.3%
AfriqueGemma-12BI1-Q3_K_M12.2B6.00 GiB2.38 GiB8.98 GiB0.02 GiB11±8.3%
granite-8b-code-instruct-4kI1-IQ3_S8.1B3.32 GiB5.06 GiB8.97 GiB0.03 GiB11±8.3%
granite-8b-code-base-4kI1-IQ3_S8.1B3.32 GiB5.06 GiB8.97 GiB0.03 GiB11±8.3%
HunyuanVideo-1.5Q8_08.3B8.38 GiB0.00 GiB8.97 GiB0.03 GiB12±8.3%
Ministral-3-8B-Instruct-2512Q3_K_S8.9B3.60 GiB4.78 GiB8.97 GiB0.03 GiB11±8.3%
Ministral-3-8B-Reasoning-2512Q3_K_S8.9B3.60 GiB4.78 GiB8.97 GiB0.03 GiB11±8.3%
Ling-mini-2.0MoEQ3_K_M16.3B7.02 GiB1.41 GiB8.97 GiB0.03 GiB25±37%
Hunyuan-7B-InstructIQ4_XS7.5B3.88 GiB4.50 GiB8.97 GiB0.03 GiB12±8.3%
zeta-2.1I1-Q3_K_M8.3B3.87 GiB4.50 GiB8.96 GiB0.04 GiB12±8.3%
gemma-4-E2B-itQ4_K_M5.1B8.16 GiB0.25 GiB8.96 GiB0.04 GiB11±8.3%
Mistral-7B-v0.2IQ4_NL7.2B3.87 GiB4.50 GiB8.96 GiB0.04 GiB12±8.3%
Qwen3.5-21B-Claude-4.6-Opus-Deckard-Heretic-Uncensored-ThinkingI1-IQ2_XS21.3B6.65 GiB1.69 GiB8.95 GiB0.05 GiB12±8.3%
Qwen3.6-21B-IQ-Ultra-Heretic-Uncensored-ThinkingI1-IQ2_XS21.3B6.65 GiB1.69 GiB8.95 GiB0.05 GiB12±8.3%
Olmo-3-7B-InstructQ3_K_M7.3B3.40 GiB4.97 GiB8.95 GiB0.05 GiB12±8.3%
Olmo-3-7B-ThinkI1-Q3_K_M7.3B3.40 GiB4.97 GiB8.95 GiB0.05 GiB12±8.3%
Bonsai-8B-unpackedIQ3_XXS8.2B3.30 GiB5.06 GiB8.95 GiB0.05 GiB12±8.3%
granite-20b-code-instruct-8kIQ3_S20.1B8.32 GiB0.00 GiB8.95 GiB0.05 GiB12±8.3%
granite-20b-code-base-8kI1-IQ3_S20.1B8.32 GiB0.00 GiB8.95 GiB0.05 GiB12±8.3%
dolphin-2.9.3-mistral-7B-32kI1-Q4_K_S7.2B3.86 GiB4.50 GiB8.95 GiB0.05 GiB12±8.3%
Mistral-7B-v0.3Q4_K_S7.2B3.86 GiB4.50 GiB8.95 GiB0.05 GiB12±8.3%
Mistral-7B-Instruct-v0.3-ParasiteI1-Q4_K_S7.2B3.86 GiB4.50 GiB8.95 GiB0.05 GiB12±8.3%
Mistral-7B-Instruct-v0.3-JbliteratedI1-Q4_K_S7.2B3.86 GiB4.50 GiB8.95 GiB0.05 GiB12±8.3%
Mistral-7B-Instruct-v0.3Q4_K_S7.2B3.86 GiB4.50 GiB8.95 GiB0.05 GiB12±8.3%
Mistral-7B-v0.3-Chinese-ChatQ4_K_S7.2B3.86 GiB4.50 GiB8.95 GiB0.05 GiB12±8.3%
mistral-7b-v0.3-bnb-4bitQ4_K_S7.5B3.86 GiB4.50 GiB8.95 GiB0.05 GiB12±8.3%
Mathstral-7B-v0.1Q4_K_S7.2B3.86 GiB4.50 GiB8.95 GiB0.05 GiB12±8.3%
Ministral-8B-Instruct-2410Q6_K8.0B6.14 GiB2.23 GiB8.95 GiB0.05 GiB12±8.3%
OLMoE-1B-7B-0924-InstructMoEI1-Q4_K_M6.9B3.92 GiB4.50 GiB8.95 GiB0.05 GiB11±37%
openchat-3.5-0106KV unresolvedQ4_K_S7.2B3.86 GiB4.50 GiB8.94 GiB0.06 GiB12±8.3%
dolphin-2.6-mistral-7bQ4_K_S7.2B3.86 GiB4.50 GiB8.94 GiB0.06 GiB12±8.3%
Silicon-Maid-7BKV unresolvedQ4_K_S7.2B3.86 GiB4.50 GiB8.94 GiB0.06 GiB12±8.3%
SciPhi-Self-RAG-Mistral-7B-32kKV unresolvedI1-Q4_K_S7.2B3.86 GiB4.50 GiB8.94 GiB0.06 GiB12±8.3%
dolphin-2.2.1-mistral-7bKV unresolvedI1-Q4_K_S7.2B3.86 GiB4.50 GiB8.94 GiB0.06 GiB12±8.3%
OpenChat-3.5-7B-Qwen-v2.0KV unresolvedI1-Q4_K_S7.2B3.86 GiB4.50 GiB8.94 GiB0.06 GiB12±8.3%
CapybaraHermes-2.5-Mistral-7BKV unresolvedQ4_K_S7.2B3.86 GiB4.50 GiB8.94 GiB0.06 GiB12±8.3%
dolphin-2.8-mistral-7b-v02Q4_K_S7.2B3.86 GiB4.50 GiB8.94 GiB0.06 GiB12±8.3%
openchat-3.5-1210KV unresolvedQ4_K_S7.2B3.86 GiB4.50 GiB8.94 GiB0.06 GiB12±8.3%
OpenHermes-2.5-Mistral-7BKV unresolvedQ4_K_S7.2B3.86 GiB4.50 GiB8.94 GiB0.06 GiB12±8.3%
Hermes-Trismegistus-Mistral-7BKV unresolvedQ4_K_S7.2B3.86 GiB4.50 GiB8.94 GiB0.06 GiB12±8.3%
Mistral-7B-OpenOrcaKV unresolvedQ4_K_S7.2B3.86 GiB4.50 GiB8.94 GiB0.06 GiB12±8.3%
dolphin-2.1-mistral-7bKV unresolvedQ4_K_S7.2B3.86 GiB4.50 GiB8.94 GiB0.06 GiB12±8.3%
OpenHermes-2-Mistral-7BKV unresolvedQ4_K_S7.2B3.86 GiB4.50 GiB8.94 GiB0.06 GiB12±8.3%
dolphin-2.6-mistral-7b-dpo-laserQ4_K_S7.2B3.86 GiB4.50 GiB8.94 GiB0.06 GiB12±8.3%
Mistral-7B-Instruct-v0.1KV unresolvedI1-Q4_K_S7.2B3.86 GiB4.50 GiB8.94 GiB0.06 GiB12±8.3%
Mistral-7B-Instruct-v0.2I1-Q4_K_S7.2B3.86 GiB4.50 GiB8.94 GiB0.06 GiB12±8.3%
ContextualKunoichi_KTO-7BI1-Q4_K_S7.2B3.86 GiB4.50 GiB8.94 GiB0.06 GiB12±8.3%
●From the file◐Predictedwhat these mean

Speed is modeled, not measured: decode is memory-bandwidth bound, so tokens per second is bytes read per token against achievable bandwidth. Mixture-of-experts models carry a wider band because only the routed experts are read each step, and few have been measured publicly.

Questions people ask

What AI models can a Apple M4 run?
1146 of 2118 indexed open-weight models fit a Apple M4 at 131,072 context with q4_0 KV cache, the largest being Qwen3-4B-Base at Q6_K. That covers text, vision-language, image, video and speech models.
How much usable memory does a Apple M4 actually have?
Its nameplate is 12 GB, but about 8.37 GiB is available to a model once driver and compositor overhead is accounted for, and only 9 GB of the pool can be allocated to the GPU at all.
Is a Apple M4 fast for local AI?
Its memory bandwidth is 120 GB/s, and that figure — not teraflops — is what governs token generation speed. Capacity decides what you can run; bandwidth decides how fast it runs.