Apple · apple

Apple M4

Apple M4 has 12 GB of unified memory at 120 GB/s — about 8.37 GiB usable after driver and compositor overhead. 1679 of 2118 indexed models fit at 4K context with q8_0 KV. Note only 9 GB of its 12 GB is allocatable to the GPU.

Spec sheet· bandwidth, theoreticalFrom the file· fit from summed bytesPredicted· speed
Memory
12 GB
LPDDR5X-7500
Bandwidth
120 GB/s
128-bit bus
Tensor FP16
—
dense
TDP
—
Context4K8K16K32K64K128K
KV cachef16q8_0q4_0quantizing the KV cache is a ~2× lever on the dominant term at long context
text 1456video 12vision language 123audio tts 21embedding 26audio asr 39image 2

What fits at 4K context

largest quantization that fits, per model · 1679 of 2118 indexed
ModelBest quantParamsWeights●KV●Total in memory◐Headroom◐tok/s◐
MN-GRAND-23.5B-Gutenberg-UNCENSORED-V2-GLM4.7-ThinkingI1-Q2_K_S23.4B7.73 GiB0.67 GiB9.00 GiB0.00 GiB11±8.3%
glm-4-9b-chat-1mQ5_K_L9.5B7.07 GiB1.33 GiB9.00 GiB0.00 GiB11±8.3%
Wan2.1-T2V-14BQ4_014.3B8.41 GiB0.00 GiB9.00 GiB0.00 GiB11±8.3%
next-8bQ8_08.2B8.11 GiB0.30 GiB8.99 GiB0.01 GiB11±8.3%
next-ocrQ8_08.8B8.11 GiB0.30 GiB8.99 GiB0.01 GiB11±8.3%
Qwen3-VL-8B-GLM-4.7-Flash-Heretic-Uncensored-ThinkingQ8_08.8B8.11 GiB0.30 GiB8.99 GiB0.01 GiB11±8.3%
Midas-FableAgent-8BQ8_08.2B8.11 GiB0.30 GiB8.99 GiB0.01 GiB11±8.3%
Qwen3-VL-8B-Heretic-1.3.0Q8_08.8B8.11 GiB0.30 GiB8.99 GiB0.01 GiB11±8.3%
Qwen3-VL-8B-ThinkingQ8_08.8B8.11 GiB0.30 GiB8.99 GiB0.01 GiB11±8.3%
Qwen3-VL-8B-Instruct-Unredacted-MAXQ8_08.8B8.11 GiB0.30 GiB8.99 GiB0.01 GiB11±8.3%
Poe-8B-GLM5-Opus4.6-Sonnet4.5-Kimi-Grok-Gemini-3-pro-preview-HERETICQ8_08.8B8.11 GiB0.30 GiB8.99 GiB0.01 GiB11±8.3%
Qwen-3-VL-8B-Instruct-hereticQ8_08.8B8.11 GiB0.30 GiB8.99 GiB0.01 GiB11±8.3%
nsfwcaption-qwen3-vl-8b-v3-safetensorsQ8_08.8B8.11 GiB0.30 GiB8.99 GiB0.01 GiB11±8.3%
Huihui-Qwen3-VL-8B-Instruct-abliteratedQ8_08.8B8.11 GiB0.30 GiB8.99 GiB0.01 GiB11±8.3%
Qwen3-VL-Reranker-8BQ8_08.8B8.11 GiB0.30 GiB8.99 GiB0.01 GiB11±8.3%
Salience-1-9BQ8_08.8B8.11 GiB0.30 GiB8.99 GiB0.01 GiB11±8.3%
Qwen3-VL-8B-InstructQ8_08.8B8.11 GiB0.30 GiB8.99 GiB0.01 GiB11±8.3%
Qwen3-VL-8B-Instruct-Uncensored-V2Q8_08.8B8.11 GiB0.30 GiB8.99 GiB0.01 GiB11±8.3%
GRaPE-2-FlashQ8_08.8B8.11 GiB0.30 GiB8.99 GiB0.01 GiB11±8.3%
Jan-v2-VL-highQ8_08.8B8.11 GiB0.30 GiB8.99 GiB0.01 GiB11±8.3%
Jan-v2-VL-medQ8_08.8B8.11 GiB0.30 GiB8.99 GiB0.01 GiB11±8.3%
DeepSeek-R1-0528-Qwen3-8BQ8_08.2B8.11 GiB0.30 GiB8.99 GiB0.01 GiB11±8.3%
Parable-Qwen3-8B-Claude-Fable-5Q8_08.2B8.11 GiB0.30 GiB8.99 GiB0.01 GiB11±8.3%
ReasonCritic-7BQ8_08.2B8.11 GiB0.30 GiB8.99 GiB0.01 GiB11±8.3%
Finch-8B-KTOQ8_08.2B8.11 GiB0.30 GiB8.99 GiB0.01 GiB11±8.3%
Finch-8BQ8_08.2B8.11 GiB0.30 GiB8.99 GiB0.01 GiB11±8.3%
mythos-9b-unhinged-hereticQ8_08.2B8.11 GiB0.30 GiB8.99 GiB0.01 GiB11±8.3%
nsfwvision-qwen3-vl-8b-v3-safetensorsQ8_08.8B8.11 GiB0.30 GiB8.99 GiB0.01 GiB11±8.3%
MathSmith-hc-Qwen3-8BQ8_08.2B8.11 GiB0.30 GiB8.99 GiB0.01 GiB11±8.3%
Qwen3-VL-8B-Thinking-Unredacted-MAXQ8_08.8B8.11 GiB0.30 GiB8.99 GiB0.01 GiB11±8.3%
MiroThinker-v1.0-8BQ8_08.2B8.11 GiB0.30 GiB8.99 GiB0.01 GiB11±8.3%
mythos-9b-unhingedQ8_08.2B8.11 GiB0.30 GiB8.99 GiB0.01 GiB11±8.3%
Maestro1-9BQ8_08.8B8.11 GiB0.30 GiB8.99 GiB0.01 GiB11±8.3%
qwen3-8b-claude-agentic-fable5Q8_08.2B8.11 GiB0.30 GiB8.99 GiB0.01 GiB11±8.3%
Ektome-Qwen3-8B-PristinelyUncensoredQ8_08.2B8.11 GiB0.30 GiB8.99 GiB0.01 GiB11±8.3%
mythos-9b-mergedQ8_08.2B8.11 GiB0.30 GiB8.99 GiB0.01 GiB11±8.3%
Qwen3-8BQ8_08.2B8.11 GiB0.30 GiB8.99 GiB0.01 GiB11±8.3%
qwen3-8b-apostateQ8_08.2B8.11 GiB0.30 GiB8.99 GiB0.01 GiB11±8.3%
Josiefied-Qwen3-8B-abliterated-v1Q8_08.2B8.11 GiB0.30 GiB8.99 GiB0.01 GiB11±8.3%
tmax-8bQ8_08.2B8.11 GiB0.30 GiB8.99 GiB0.01 GiB11±8.3%
Qwen3-8B-abliteratedQ8_08.2B8.11 GiB0.30 GiB8.99 GiB0.01 GiB11±8.3%
Marco-DeepResearch-8BQ8_08.2B8.11 GiB0.30 GiB8.99 GiB0.01 GiB11±8.3%
Qwen3-8B-UncensoredQ8_08.2B8.11 GiB0.30 GiB8.99 GiB0.01 GiB11±8.3%
LMT-60-8BQ8_08.2B8.11 GiB0.30 GiB8.99 GiB0.01 GiB11±8.3%
Nemotron-Orchestrator-8BQ8_08.2B8.11 GiB0.30 GiB8.99 GiB0.01 GiB11±8.3%
Huihui-Qwen3-8B-abliterated-v2Q8_08.2B8.11 GiB0.30 GiB8.99 GiB0.01 GiB11±8.3%
gemma-4-A4B-98e-v7-coder-itMoEQ2_K20.5B8.22 GiB0.24 GiB8.99 GiB0.01 GiB11±8.3%
gemma-4-A4B-98e-v7-coderx-itMoEQ2_K20.5B8.22 GiB0.24 GiB8.99 GiB0.01 GiB11±8.3%
T-lite-it-2.1Q8_08.2B8.11 GiB0.30 GiB8.99 GiB0.01 GiB11±8.3%
MiniCPM-o-4_5Q8_09.4B8.11 GiB0.30 GiB8.99 GiB0.01 GiB11±8.3%
Qwen3-Reranker-8BQ8_08.2B8.11 GiB0.30 GiB8.99 GiB0.01 GiB11±8.3%
Bonsai-8B-unpackedQ8_08.2B8.11 GiB0.30 GiB8.99 GiB0.01 GiB11±8.3%
INTELLECT-1-InstructQ6_K_L10.2B8.05 GiB0.35 GiB8.98 GiB0.02 GiB11±8.3%
Qwen3-15B-A2B-BaseMoEQ4_K_S15.6B8.33 GiB0.10 GiB8.98 GiB0.02 GiB41±37%
medgemma-27b-itI1-IQ2_XS28.8B7.86 GiB0.49 GiB8.98 GiB0.02 GiB12±8.3%
gemma-3-27b-it-abliterated-refined-visionI1-IQ2_XS27.4B7.86 GiB0.49 GiB8.98 GiB0.02 GiB12±8.3%
Nidum-Gemma-3-27B-it-UncensoredI1-IQ2_XS27.4B7.86 GiB0.49 GiB8.98 GiB0.02 GiB12±8.3%
gemma-3-27b-it-abliteratedIQ2_XS27.4B7.86 GiB0.49 GiB8.98 GiB0.02 GiB12±8.3%
AtomicGPT-gemma3-27bI1-IQ2_XS27.4B7.86 GiB0.49 GiB8.98 GiB0.02 GiB12±8.3%
Unbound-v1.12.0-27BI1-IQ2_XS27.4B7.86 GiB0.49 GiB8.98 GiB0.02 GiB12±8.3%
●From the file◐Predictedwhat these mean

Speed is modeled, not measured: decode is memory-bandwidth bound, so tokens per second is bytes read per token against achievable bandwidth. Mixture-of-experts models carry a wider band because only the routed experts are read each step, and few have been measured publicly.

Questions people ask

What AI models can a Apple M4 run?
1679 of 2118 indexed open-weight models fit a Apple M4 at 4,096 context with q8_0 KV cache, the largest being MN-GRAND-23.5B-Gutenberg-UNCENSORED-V2-GLM4.7-Thinking at I1-Q2_K_S. That covers text, vision-language, image, video and speech models.
How much usable memory does a Apple M4 actually have?
Its nameplate is 12 GB, but about 8.37 GiB is available to a model once driver and compositor overhead is accounted for, and only 9 GB of the pool can be allocated to the GPU at all.
Is a Apple M4 fast for local AI?
Its memory bandwidth is 120 GB/s, and that figure — not teraflops — is what governs token generation speed. Capacity decides what you can run; bandwidth decides how fast it runs.
Apple M4 — what AI models can it run locally? — ossmodeldb