Apple · apple

Apple M4

Apple M4 has 12 GB of unified memory at 120 GB/s — about 8.37 GiB usable after driver and compositor overhead. 784 of 2118 indexed models fit at 128K context with q8_0 KV. Note only 9 GB of its 12 GB is allocatable to the GPU.

Spec sheet· bandwidth, theoreticalFrom the file· fit from summed bytesPredicted· speed
Memory
12 GB
LPDDR5X-7500
Bandwidth
120 GB/s
128-bit bus
Tensor FP16
—
dense
TDP
—
Context4K8K16K32K64K128K
KV cachef16q8_0q4_0quantizing the KV cache is a ~2× lever on the dominant term at long context
text 609video 12vision language 86audio tts 19embedding 21audio asr 36image 1

What fits at 128K context

largest quantization that fits, per model · 784 of 2118 indexed
ModelBest quantParamsWeights●KV●Total in memory◐Headroom◐tok/s◐
granite-vision-3.3-2bQ8_03.0B3.14 GiB5.31 GiB9.00 GiB0.00 GiB11±8.3%
Wan2.1-T2V-14BQ4_014.3B8.41 GiB0.00 GiB9.00 GiB0.00 GiB11±8.3%
Felldude-Uncensored-Ministral3-3B-bf16I1-Q3_K_S3.8B1.53 GiB6.91 GiB9.00 GiB0.00 GiB11±8.3%
Ministral-3-3B-Instruct-2512-BF16Q3_K_S4.3B1.53 GiB6.91 GiB9.00 GiB0.00 GiB11±8.3%
Amaretto-3BI1-Q3_K_S4.3B1.53 GiB6.91 GiB9.00 GiB0.00 GiB11±8.3%
Ministral-3-3B-Instruct-2512Q3_K_S3.8B1.53 GiB6.91 GiB9.00 GiB0.00 GiB11±8.3%
Ministral-3-3B-Reasoning-2512Q3_K_S4.3B1.53 GiB6.91 GiB9.00 GiB0.00 GiB11±8.3%
gemma-4-E4B-it-Uncensored-MAXQ8_08.0B7.46 GiB0.97 GiB8.99 GiB0.01 GiB11±8.3%
GLM-4.6V-FlashQ4_K_M10.3B5.74 GiB2.66 GiB8.99 GiB0.01 GiB11±8.3%
GLM-Z1-9B-0414Q4_K_M9.4B5.74 GiB2.66 GiB8.99 GiB0.01 GiB11±8.3%
glm4.1v-9b-base-sftI1-Q4_K_M10.3B5.74 GiB2.66 GiB8.99 GiB0.01 GiB11±8.3%
GLM-4-9B-0414Q4_K_M9.4B5.74 GiB2.66 GiB8.99 GiB0.01 GiB11±8.3%
GLM-4.1V-9B-ThinkingQ4_K_M10.3B5.74 GiB2.66 GiB8.99 GiB0.01 GiB11±8.3%
internlm3-8b-instructQ4_18.8B5.22 GiB3.19 GiB8.99 GiB0.01 GiB11±8.3%
OpenClaude-1.7B-MergedQ4_K_S1.7B1.00 GiB7.44 GiB8.99 GiB0.01 GiB11±8.3%
EVA-Yi-1.5-9B-32K-V1I1-IQ1_M8.8B2.03 GiB6.38 GiB8.98 GiB0.02 GiB11±8.3%
Yi-Coder-9B-ChatIQ1_M8.8B2.03 GiB6.38 GiB8.98 GiB0.02 GiB11±8.3%
Llama-3.2-3BTQ2_03.2B0.99 GiB7.44 GiB8.98 GiB0.02 GiB11±8.3%
Qwen3-1.7BQ3_K_M2.0B1.00 GiB7.44 GiB8.98 GiB0.02 GiB11±8.3%
AfriqueGemma-12BI1-IQ2_XS12.2B3.88 GiB4.50 GiB8.98 GiB0.02 GiB11±8.3%
InternVL3_5-14BQ4_K_M15.1B8.38 GiB0.00 GiB8.98 GiB0.02 GiB11±8.3%
Qwen3.5-9BQ5_K_M9.7B6.27 GiB2.13 GiB8.98 GiB0.02 GiB11±8.3%
Llama-3.2-3B-Instruct-abliteratedI1-IQ1_M3.6B0.98 GiB7.44 GiB8.98 GiB0.02 GiB11±8.3%
HunyuanVideo-1.5Q8_08.3B8.38 GiB0.00 GiB8.97 GiB0.03 GiB12±8.3%
Llama-3.2-3B-InstructUD-IQ2_XXS3.2B0.97 GiB7.44 GiB8.97 GiB0.03 GiB11±8.3%
Supertron2-Reranker-2BI1-Q4_K_S2.1B0.99 GiB7.44 GiB8.97 GiB0.03 GiB11±8.3%
Uni-MuMER-Qwen3-VL-2BI1-Q4_K_S2.1B0.99 GiB7.44 GiB8.97 GiB0.03 GiB11±8.3%
Qwen3-VL-2B-ThinkingQ4_K_S2.1B0.99 GiB7.44 GiB8.97 GiB0.03 GiB11±8.3%
Qwen3-VL-Reranker-2BI1-Q4_K_S2.1B0.99 GiB7.44 GiB8.97 GiB0.03 GiB11±8.3%
Qwen3-VL-2B-InstructQ4_K_S2.1B0.99 GiB7.44 GiB8.97 GiB0.03 GiB11±8.3%
Qwen3-VL-Embedding-2BQ4_K_S2.1B0.99 GiB7.44 GiB8.97 GiB0.03 GiB11±8.3%
OpenCaption-2B-VL-SFT-v1.0I1-Q4_K_S2.1B0.99 GiB7.44 GiB8.97 GiB0.03 GiB11±8.3%
Atomight-V2.5-1.7BI1-Q4_K_S1.7B0.99 GiB7.44 GiB8.97 GiB0.03 GiB11±8.3%
gaon-1.7b-v2-translateI1-Q4_K_S1.7B0.99 GiB7.44 GiB8.97 GiB0.03 GiB11±8.3%
gaon-1.7b-v2-instructI1-Q4_K_S1.7B0.99 GiB7.44 GiB8.97 GiB0.03 GiB11±8.3%
Lightning-1.7BQ4_K_S1.7B0.99 GiB7.44 GiB8.97 GiB0.03 GiB11±8.3%
DorsetHeatwaveLLM2I1-Q4_K_S1.7B0.99 GiB7.44 GiB8.97 GiB0.03 GiB11±8.3%
granite-3.1-2b-instructQ4_K2.5B3.10 GiB5.31 GiB8.96 GiB0.04 GiB11±8.3%
gemma-4-E4B-uncensoredQ8_07.9B7.43 GiB0.97 GiB8.96 GiB0.04 GiB11±8.3%
gemma-4-E4B-it-qat-heretic_decensoredQ8_07.9B7.43 GiB0.97 GiB8.96 GiB0.04 GiB11±8.3%
gemma-4-E4B-it-QAT-SOMPOA-heresyQ8_07.9B7.43 GiB0.97 GiB8.96 GiB0.04 GiB11±8.3%
Tinman-gemma4-companion-mergedQ8_07.9B7.43 GiB0.97 GiB8.96 GiB0.04 GiB11±8.3%
starcoder2-7bKV unresolvedQ4_K_M7.2B4.10 GiB4.25 GiB8.96 GiB0.04 GiB12±8.3%
orpheus-3b-0.1-ftUD-IQ1_M3.8B0.95 GiB7.44 GiB8.95 GiB0.05 GiB11±8.3%
granite-20b-code-instruct-8kIQ3_S20.1B8.32 GiB0.00 GiB8.95 GiB0.05 GiB12±8.3%
granite-20b-code-base-8kI1-IQ3_S20.1B8.32 GiB0.00 GiB8.95 GiB0.05 GiB12±8.3%
Llama-Doctor-3.2-3B-InstructI1-IQ2_XXS3.2B0.95 GiB7.44 GiB8.94 GiB0.06 GiB11±8.3%
Llama-3.2-3B-Instruct-roleplay-tunedI1-IQ2_XXS3.2B0.95 GiB7.44 GiB8.94 GiB0.06 GiB11±8.3%
Llama-3.2-3B-Instruct-heretic-ablitered-uncensoredI1-IQ2_XXS3.2B0.95 GiB7.44 GiB8.94 GiB0.06 GiB11±8.3%
Llama3.2-3B-creative-writer-v0.1I1-IQ2_XXS3.2B0.95 GiB7.44 GiB8.94 GiB0.06 GiB11±8.3%
Firefly-V3.2I1-IQ2_XXS3.2B0.95 GiB7.44 GiB8.94 GiB0.06 GiB11±8.3%
Firefly-V3I1-IQ2_XXS3.2B0.95 GiB7.44 GiB8.94 GiB0.06 GiB11±8.3%
Ministral-8B-Instruct-2410IQ4_XS8.0B4.14 GiB4.21 GiB8.94 GiB0.06 GiB12±8.3%
Ling-mini-2.0MoEQ2_K_L16.3B5.74 GiB2.66 GiB8.93 GiB0.07 GiB17±37%
nomic-embed-codeQ5_K_S7.1B4.60 GiB3.72 GiB8.92 GiB0.08 GiB12±8.3%
Fara1.5-9BQ4_K_L9.4B6.21 GiB2.13 GiB8.92 GiB0.08 GiB12±8.3%
QwenPaw-Flash-9BQ4_K_L9.4B6.21 GiB2.13 GiB8.92 GiB0.08 GiB12±8.3%
grug-9bQ4_K_L9.4B6.21 GiB2.13 GiB8.92 GiB0.08 GiB12±8.3%
OmniCoder-9BQ4_K_L9.4B6.21 GiB2.13 GiB8.92 GiB0.08 GiB12±8.3%
Ornith-1.0-9BQ4_K_L9.2B6.21 GiB2.13 GiB8.92 GiB0.08 GiB12±8.3%
●From the file◐Predictedwhat these mean

Speed is modeled, not measured: decode is memory-bandwidth bound, so tokens per second is bytes read per token against achievable bandwidth. Mixture-of-experts models carry a wider band because only the routed experts are read each step, and few have been measured publicly.

Questions people ask

What AI models can a Apple M4 run?
784 of 2118 indexed open-weight models fit a Apple M4 at 131,072 context with q8_0 KV cache, the largest being granite-vision-3.3-2b at Q8_0. That covers text, vision-language, image, video and speech models.
How much usable memory does a Apple M4 actually have?
Its nameplate is 12 GB, but about 8.37 GiB is available to a model once driver and compositor overhead is accounted for, and only 9 GB of the pool can be allocated to the GPU at all.
Is a Apple M4 fast for local AI?
Its memory bandwidth is 120 GB/s, and that figure — not teraflops — is what governs token generation speed. Capacity decides what you can run; bandwidth decides how fast it runs.