Apple · apple

Apple M4

Apple M4 has 12 GB of unified memory at 120 GB/s — about 8.37 GiB usable after driver and compositor overhead. 1196 of 2118 indexed models fit at 64K context with q8_0 KV. Note only 9 GB of its 12 GB is allocatable to the GPU.

Spec sheet· bandwidth, theoreticalFrom the file· fit from summed bytesPredicted· speed
Memory
12 GB
LPDDR5X-7500
Bandwidth
120 GB/s
128-bit bus
Tensor FP16
—
dense
TDP
—
Context4K8K16K32K64K128K
KV cachef16q8_0q4_0quantizing the KV cache is a ~2× lever on the dominant term at long context
text 1000video 12vision language 99embedding 26audio asr 38audio tts 20image 1

What fits at 64K context

largest quantization that fits, per model · 1196 of 2118 indexed
ModelBest quantParamsWeights●KV●Total in memory◐Headroom◐tok/s◐
Llama-3.1-8B-InstructIQ4_XS8.0B4.16 GiB4.25 GiB9.00 GiB0.00 GiB11±8.3%
Llama-3.1-Nemotron-Nano-8B-v1IQ4_XS8.0B4.16 GiB4.25 GiB9.00 GiB0.00 GiB11±8.3%
DeepSeek-R1-Distill-Llama-8BIQ4_XS8.0B4.16 GiB4.25 GiB9.00 GiB0.00 GiB11±8.3%
Mistral-NeMo-Minitron-8B-InstructQ2_K8.4B3.10 GiB5.31 GiB9.00 GiB0.00 GiB11±8.3%
xLAM-7b-rQ4_K_L7.2B4.16 GiB4.25 GiB9.00 GiB0.00 GiB11±8.3%
MegaBeam-Mistral-7B-512kQ4_K_L7.2B4.16 GiB4.25 GiB9.00 GiB0.00 GiB11±8.3%
Wan2.1-T2V-14BQ4_014.3B8.41 GiB0.00 GiB9.00 GiB0.00 GiB11±8.3%
next-8bI1-IQ3_M8.2B3.63 GiB4.78 GiB8.99 GiB0.01 GiB11±8.3%
Supertron2-Reranker-8BI1-IQ3_M8.8B3.63 GiB4.78 GiB8.99 GiB0.01 GiB11±8.3%
next-ocrI1-IQ3_M8.8B3.63 GiB4.78 GiB8.99 GiB0.01 GiB11±8.3%
Qwen3-VL-8B-GLM-4.7-Flash-Heretic-Uncensored-ThinkingI1-IQ3_M8.8B3.63 GiB4.78 GiB8.99 GiB0.01 GiB11±8.3%
Midas-FableAgent-8BI1-IQ3_M8.2B3.63 GiB4.78 GiB8.99 GiB0.01 GiB11±8.3%
Qwen3-VL-8B-Heretic-1.3.0I1-IQ3_M8.8B3.63 GiB4.78 GiB8.99 GiB0.01 GiB11±8.3%
Qwen3-VL-8B-Thinking-Unredacted-MAXI1-IQ3_M8.8B3.63 GiB4.78 GiB8.99 GiB0.01 GiB11±8.3%
Qwen3-VL-8B-Instruct-Minecraft-MT-en-zhI1-IQ3_M8.8B3.63 GiB4.78 GiB8.99 GiB0.01 GiB11±8.3%
Qwen-3-VL-8B-Instruct-hereticI1-IQ3_M8.8B3.63 GiB4.78 GiB8.99 GiB0.01 GiB11±8.3%
Poe-8B-GLM5-Opus4.6-Sonnet4.5-Kimi-Grok-Gemini-3-pro-preview-HERETICI1-IQ3_M8.8B3.63 GiB4.78 GiB8.99 GiB0.01 GiB11±8.3%
ToolCUA-8BI1-IQ3_M8.8B3.63 GiB4.78 GiB8.99 GiB0.01 GiB11±8.3%
Huihui-Qwen3-VL-8B-Instruct-abliteratedI1-IQ3_M8.8B3.63 GiB4.78 GiB8.99 GiB0.01 GiB11±8.3%
Qwen3-VL-Reranker-8BI1-IQ3_M8.8B3.63 GiB4.78 GiB8.99 GiB0.01 GiB11±8.3%
Salience-1-9BI1-IQ3_M8.8B3.63 GiB4.78 GiB8.99 GiB0.01 GiB11±8.3%
Qwen3-VL-8B-Instruct-Uncensored-V2I1-IQ3_M8.8B3.63 GiB4.78 GiB8.99 GiB0.01 GiB11±8.3%
Maestro1-9BI1-IQ3_M8.8B3.63 GiB4.78 GiB8.99 GiB0.01 GiB11±8.3%
GRaPE-2-FlashI1-IQ3_M8.8B3.63 GiB4.78 GiB8.99 GiB0.01 GiB11±8.3%
Jan-v2-VL-medI1-IQ3_M8.8B3.63 GiB4.78 GiB8.99 GiB0.01 GiB11±8.3%
Parable-Qwen3-8B-Claude-Fable-5I1-IQ3_M8.2B3.63 GiB4.78 GiB8.99 GiB0.01 GiB11±8.3%
ReasonCritic-7BI1-IQ3_M8.2B3.63 GiB4.78 GiB8.99 GiB0.01 GiB11±8.3%
mythos-9b-unhinged-hereticI1-IQ3_M8.2B3.63 GiB4.78 GiB8.99 GiB0.01 GiB11±8.3%
Finch-8B-KTOI1-IQ3_M8.2B3.63 GiB4.78 GiB8.99 GiB0.01 GiB11±8.3%
Finch-8BI1-IQ3_M8.2B3.63 GiB4.78 GiB8.99 GiB0.01 GiB11±8.3%
MathSmith-hc-Qwen3-8BI1-IQ3_M8.2B3.63 GiB4.78 GiB8.99 GiB0.01 GiB11±8.3%
MiroThinker-v1.0-8BI1-IQ3_M8.2B3.63 GiB4.78 GiB8.99 GiB0.01 GiB11±8.3%
mythos-9b-unhingedI1-IQ3_M8.2B3.63 GiB4.78 GiB8.99 GiB0.01 GiB11±8.3%
Ektome-Qwen3-8B-PristinelyUncensoredI1-IQ3_M8.2B3.63 GiB4.78 GiB8.99 GiB0.01 GiB11±8.3%
Marco-DeepResearch-8BI1-IQ3_M8.2B3.63 GiB4.78 GiB8.99 GiB0.01 GiB11±8.3%
mythos-9b-mergedI1-IQ3_M8.2B3.63 GiB4.78 GiB8.99 GiB0.01 GiB11±8.3%
qwen3-8b-apostateI1-IQ3_M8.2B3.63 GiB4.78 GiB8.99 GiB0.01 GiB11±8.3%
Josiefied-Qwen3-8B-abliterated-v1I1-IQ3_M8.2B3.63 GiB4.78 GiB8.99 GiB0.01 GiB11±8.3%
tmax-8bI1-IQ3_M8.2B3.63 GiB4.78 GiB8.99 GiB0.01 GiB11±8.3%
Qwen3-8B-abliteratedIQ3_M8.2B3.63 GiB4.78 GiB8.99 GiB0.01 GiB11±8.3%
Qwen3-8BIQ3_M8.2B3.63 GiB4.78 GiB8.99 GiB0.01 GiB11±8.3%
Qwen3-8B-DeepSeek-v3.2-Speciale-DistillIQ3_M8.2B3.63 GiB4.78 GiB8.99 GiB0.01 GiB11±8.3%
story_generation_Qwen3_8B_RLI1-IQ3_M8.2B3.63 GiB4.78 GiB8.99 GiB0.01 GiB11±8.3%
AReaL-boba-2-8B-OpenI1-IQ3_M8.2B3.63 GiB4.78 GiB8.99 GiB0.01 GiB11±8.3%
DS-R1-Qwen3-8B-ArliAI-RpR-v4-SmallI1-IQ3_M8.2B3.63 GiB4.78 GiB8.99 GiB0.01 GiB11±8.3%
Nemotron-Orchestrator-8BIQ3_M8.2B3.63 GiB4.78 GiB8.99 GiB0.01 GiB11±8.3%
DeepSeek-R1-0528-Qwen3-8BIQ3_M8.2B3.63 GiB4.78 GiB8.99 GiB0.01 GiB11±8.3%
S1-Base-8BI1-IQ3_M8.2B3.63 GiB4.78 GiB8.99 GiB0.01 GiB11±8.3%
Huihui-Qwen3-8B-abliterated-v2I1-IQ3_M8.2B3.63 GiB4.78 GiB8.99 GiB0.01 GiB11±8.3%
Step3-VL-10B-BaseI1-IQ3_M10.2B3.63 GiB4.78 GiB8.99 GiB0.01 GiB11±8.3%
zeta-2.1I1-Q3_K_L8.3B4.15 GiB4.25 GiB8.99 GiB0.01 GiB11±8.3%
Qwen3-Reranker-8BI1-IQ3_M8.2B3.63 GiB4.78 GiB8.99 GiB0.01 GiB11±8.3%
Jan-code-4bQ6_K_L4.4B3.65 GiB4.78 GiB8.99 GiB0.01 GiB11±8.3%
granite-3.1-8b-instructIQ3_XXS8.2B3.10 GiB5.31 GiB8.99 GiB0.01 GiB11±8.3%
Assistant_Pepe_8BIQ4_XS—4.15 GiB4.25 GiB8.99 GiB0.01 GiB11±8.3%
Luna-7B-A4BMoEI1-Q4_K_S6.7B3.64 GiB4.78 GiB8.98 GiB0.02 GiB8±37%
Bonsai-8B-unpackedQ3_K_S8.2B3.62 GiB4.78 GiB8.98 GiB0.02 GiB11±8.3%
salamandra-7b-instruct-2606I1-IQ4_XS7.8B4.15 GiB4.25 GiB8.98 GiB0.02 GiB11±8.3%
Foundation-Sec-8B-InstructI1-IQ4_XS8.0B4.14 GiB4.25 GiB8.98 GiB0.02 GiB11±8.3%
Foundation-Sec-8B-Instruct-hereticI1-IQ4_XS8.0B4.14 GiB4.25 GiB8.98 GiB0.02 GiB11±8.3%
●From the file◐Predictedwhat these mean

Speed is modeled, not measured: decode is memory-bandwidth bound, so tokens per second is bytes read per token against achievable bandwidth. Mixture-of-experts models carry a wider band because only the routed experts are read each step, and few have been measured publicly.

Questions people ask

What AI models can a Apple M4 run?
1196 of 2118 indexed open-weight models fit a Apple M4 at 65,536 context with q8_0 KV cache, the largest being Llama-3.1-8B-Instruct at IQ4_XS. That covers text, vision-language, image, video and speech models.
How much usable memory does a Apple M4 actually have?
Its nameplate is 12 GB, but about 8.37 GiB is available to a model once driver and compositor overhead is accounted for, and only 9 GB of the pool can be allocated to the GPU at all.
Is a Apple M4 fast for local AI?
Its memory bandwidth is 120 GB/s, and that figure — not teraflops — is what governs token generation speed. Capacity decides what you can run; bandwidth decides how fast it runs.
Apple M4 — what AI models can it run locally? — ossmodeldb