Apple · apple

Apple M3

Apple M3 has 8 GB of unified memory at 102 GB/s — about 5.58 GiB usable after driver and compositor overhead. 1271 of 2118 indexed models fit at 4K context with f16 KV. Note only 6 GB of its 8 GB is allocatable to the GPU.

Spec sheet· bandwidth, theoreticalFrom the file· fit from summed bytesPredicted· speed
Memory
8 GB
LPDDR5-6400
Bandwidth
102 GB/s
128-bit bus
Tensor FP16
—
dense
TDP
—
Context4K8K16K32K64K128K
KV cachef16q8_0q4_0quantizing the KV cache is a ~2× lever on the dominant term at long context
text 1086image 1vision language 96audio asr 38embedding 26audio tts 19video 5

What fits at 4K context

largest quantization that fits, per model · 1271 of 2118 indexed
ModelBest quantParamsWeights●KV●Total in memory◐Headroom◐tok/s◐
Gemma-4-E4B-LuchadorQ5_K_S8.0B5.31 GiB0.12 GiB6.00 GiB0.00 GiB15±8.3%
Mistral-NeMo-Minitron-8B-InstructQ4_K_M8.4B4.79 GiB0.63 GiB6.00 GiB0.00 GiB15±8.3%
Fimbulvetr-11B-v2I1-IQ3_M10.7B4.66 GiB0.75 GiB5.99 GiB0.01 GiB15±8.3%
LFM2-24B-A2BMoEIQ2_XXS23.8B5.35 GiB0.08 GiB5.99 GiB0.01 GiB49±37%
Janus-Pro-7BI1-IQ4_XS7.4B3.54 GiB1.88 GiB5.99 GiB0.01 GiB15±8.3%
deepseek-coder-7b-instruct-v1.5I1-IQ4_XS6.9B3.54 GiB1.88 GiB5.99 GiB0.01 GiB15±8.3%
Bonsai-8B-unpackedQ4_K_M8.2B4.84 GiB0.56 GiB5.99 GiB0.01 GiB15±8.3%
internlm3-8b-instructQ4_18.8B5.22 GiB0.19 GiB5.99 GiB0.01 GiB15±8.3%
Gemma-4-E4B-it-Minecraft-MT-en-zh-v0.1I1-Q5_K_S8.0B5.30 GiB0.12 GiB5.98 GiB0.02 GiB15±8.3%
gemma-4-E4B-Queen-it-qat-q4_0-unquantizedI1-Q5_K_S8.0B5.30 GiB0.12 GiB5.98 GiB0.02 GiB15±8.3%
Gemma-4-E4B-Luchador-RudoI1-Q5_K_S8.0B5.30 GiB0.12 GiB5.98 GiB0.02 GiB15±8.3%
supergemma4-e4b-abliteratedI1-Q5_K_S7.5B5.30 GiB0.12 GiB5.98 GiB0.02 GiB15±8.3%
Gemma-4-E4B-AbliteratedI1-Q5_K_S8.0B5.30 GiB0.12 GiB5.98 GiB0.02 GiB15±8.3%
gemma-4-E4B-it-ultra-uncensored-hereticQ5_K_S8.0B5.30 GiB0.12 GiB5.98 GiB0.02 GiB15±8.3%
gemma-4-E4B-it-The-DECKARD-Claude-Opus-Expresso-Universe-HERETIC-UNCENSORED-ThinkingI1-Q5_K_S8.0B5.30 GiB0.12 GiB5.98 GiB0.02 GiB15±8.3%
gemma-4-E4B-it-The-DECKARD-Expresso-Universe-HERETIC-UNCENSORED-ThinkingI1-Q5_K_S8.0B5.30 GiB0.12 GiB5.98 GiB0.02 GiB15±8.3%
gemma-4-E4B-it-hereticI1-Q5_K_S8.0B5.30 GiB0.12 GiB5.98 GiB0.02 GiB15±8.3%
gemma-4-E4B-it-Claude-Opus-4.5-HERETIC-UNCENSORED-ThinkingI1-Q5_K_S8.0B5.30 GiB0.12 GiB5.98 GiB0.02 GiB15±8.3%
Huihui-gemma-4-E4B-it-abliteratedI1-Q5_K_S8.0B5.30 GiB0.12 GiB5.98 GiB0.02 GiB15±8.3%
gemma-4-E4B-it-Uncensored-MAXI1-Q5_K_S8.0B5.30 GiB0.12 GiB5.98 GiB0.02 GiB15±8.3%
Darkidol-Gemma-4-E4B-itI1-Q5_K_S8.0B5.30 GiB0.12 GiB5.98 GiB0.02 GiB15±8.3%
gemma-4-E4B-it-abliteratedI1-Q5_K_S8.0B5.30 GiB0.12 GiB5.98 GiB0.02 GiB15±8.3%
OpenMedResearch-Gemma-4E4NI1-Q5_K_S8.0B5.30 GiB0.12 GiB5.98 GiB0.02 GiB15±8.3%
Reasoning-Medical0.1-E4B-sftI1-Q5_K_S8.0B5.30 GiB0.12 GiB5.98 GiB0.02 GiB15±8.3%
gemma-4-E4BQ5_K_S8.0B5.30 GiB0.12 GiB5.98 GiB0.02 GiB15±8.3%
gemma-4-12BIQ3_XXS12.0B4.67 GiB0.72 GiB5.98 GiB0.02 GiB15±8.3%
codegeex4-all-9bIQ1_S9.4B2.89 GiB2.50 GiB5.98 GiB0.02 GiB15±8.3%
Cydonia-v1.3-Magnum-v4-22BI1-IQ1_S22.2B4.50 GiB0.88 GiB5.98 GiB0.02 GiB15±8.3%
Mistral-Small-22B-ArliAI-RPMax-v1.1I1-IQ1_S22.2B4.50 GiB0.88 GiB5.98 GiB0.02 GiB15±8.3%
magnum-v4-22bI1-IQ1_S22.2B4.50 GiB0.88 GiB5.98 GiB0.02 GiB15±8.3%
Codestral-22B-v0.1IQ1_S22.2B4.50 GiB0.88 GiB5.98 GiB0.02 GiB15±8.3%
Codestral-22B-v0.1-hfIQ1_S22.2B4.50 GiB0.88 GiB5.98 GiB0.02 GiB15±8.3%
gemma-3-12b-it-abliteratedQ2_K_L12.2B4.67 GiB0.72 GiB5.98 GiB0.02 GiB15±8.3%
Teuken-7B-instruct-research-v0.4I1-Q5_K_M7.5B5.27 GiB0.13 GiB5.98 GiB0.02 GiB15±8.3%
dolphin-2.9.1-mixtral-1x22bMoEI1-IQ1_S22.2B4.49 GiB0.88 GiB5.98 GiB0.02 GiB9±37%
glm-4-9b-chatIQ1_S9.4B2.89 GiB2.50 GiB5.98 GiB0.02 GiB15±8.3%
Hunyuan-7B-InstructQ5_07.5B4.89 GiB0.50 GiB5.98 GiB0.02 GiB15±8.3%
rnj-1-instructQ4_K_L8.3B4.88 GiB0.50 GiB5.98 GiB0.02 GiB15±8.3%
OLMo-2-1124-7B-InstructQ3_K_M7.3B3.40 GiB2.00 GiB5.98 GiB0.02 GiB15±8.3%
Olmo-3-7B-InstructQ3_K_M7.3B3.40 GiB2.00 GiB5.98 GiB0.02 GiB15±8.3%
Olmo-3-7B-ThinkI1-Q3_K_M7.3B3.40 GiB2.00 GiB5.98 GiB0.02 GiB15±8.3%
ERNIE-4.5-21B-A3B-ThinkingIQ2_XXS21.8B5.19 GiB0.22 GiB5.98 GiB0.02 GiB15±8.3%
Qwen3.6-12B-IQ-Ultra-Heretic-Uncensored-Thinking-V2-HightopIQ3_S12.1B5.27 GiB0.09 GiB5.98 GiB0.02 GiB15±8.3%
AMALIA-9B-0626-DPOIQ4_XS9.2B4.74 GiB0.66 GiB5.97 GiB0.03 GiB15±8.3%
Parable-Granite-4.1-8B-Claude-Fable-5I1-Q4_K_M8.4B4.77 GiB0.63 GiB5.97 GiB0.03 GiB15±8.3%
Swallow-7b-NVE-instruct-hfIQ4_XS6.7B3.40 GiB2.00 GiB5.97 GiB0.03 GiB15±8.3%
NVIDIA-Nemotron-Nano-12B-v2Q2_K12.3B4.38 GiB0.97 GiB5.97 GiB0.03 GiB15±8.3%
Aya-Medikal-V2I1-Q4_18.0B4.87 GiB0.50 GiB5.97 GiB0.03 GiB15±8.3%
Hy-MT2-7BQ5_K_S8.0B4.88 GiB0.50 GiB5.97 GiB0.03 GiB15±8.3%
Qwen3.5-9BIQ4_NL9.7B5.26 GiB0.13 GiB5.97 GiB0.03 GiB15±8.3%
Phi-3-mini-4k-instructKV unresolvedIQ2_XS3.8B3.91 GiB1.50 GiB5.97 GiB0.03 GiB15±8.3%
granite-3.1-8b-instructQ4_18.2B4.76 GiB0.63 GiB5.97 GiB0.03 GiB15±8.3%
Marco-Nano-InstructMoEI1-Q4_K_M8.0B5.00 GiB0.44 GiB5.96 GiB0.04 GiB42±37%
Ministral-3-8B-Instruct-2512Q4_K_M8.9B4.84 GiB0.53 GiB5.96 GiB0.04 GiB15±8.3%
Ministral-3-8B-Reasoning-2512Q4_K_M8.9B4.84 GiB0.53 GiB5.96 GiB0.04 GiB15±8.3%
Ministral-3-8B-Instruct-2512-BF16-abliteratedI1-Q4_K_M8.9B4.84 GiB0.53 GiB5.96 GiB0.04 GiB15±8.3%
Amaretto-8BI1-Q4_K_M8.9B4.84 GiB0.53 GiB5.96 GiB0.04 GiB15±8.3%
Ministral-3-8B-Reasoning-2512-hereticQ4_K_M8.9B4.84 GiB0.53 GiB5.96 GiB0.04 GiB15±8.3%
INTELLECT-1-InstructI1-Q3_K_M10.2B4.71 GiB0.66 GiB5.96 GiB0.04 GiB15±8.3%
LFM2-8B-A1BMoEQ5_K_S8.3B5.37 GiB0.05 GiB5.96 GiB0.04 GiB39±37%
●From the file◐Predictedwhat these mean

Speed is modeled, not measured: decode is memory-bandwidth bound, so tokens per second is bytes read per token against achievable bandwidth. Mixture-of-experts models carry a wider band because only the routed experts are read each step, and few have been measured publicly.

Questions people ask

What AI models can a Apple M3 run?
1271 of 2118 indexed open-weight models fit a Apple M3 at 4,096 context with f16 KV cache, the largest being Gemma-4-E4B-Luchador at Q5_K_S. That covers text, vision-language, image, video and speech models.
How much usable memory does a Apple M3 actually have?
Its nameplate is 8 GB, but about 5.58 GiB is available to a model once driver and compositor overhead is accounted for, and only 6 GB of the pool can be allocated to the GPU at all.
Is a Apple M3 fast for local AI?
Its memory bandwidth is 102 GB/s, and that figure — not teraflops — is what governs token generation speed. Capacity decides what you can run; bandwidth decides how fast it runs.