Apple · apple

Apple M3

Apple M3 has 8 GB of unified memory at 102 GB/s — about 5.58 GiB usable after driver and compositor overhead. 1050 of 2118 indexed models fit at 32K context with q8_0 KV. Note only 6 GB of its 8 GB is allocatable to the GPU.

Spec sheet· bandwidth, theoreticalFrom the file· fit from summed bytesPredicted· speed
Memory
8 GB
LPDDR5-6400
Bandwidth
102 GB/s
128-bit bus
Tensor FP16
—
dense
TDP
—
Context4K8K16K32K64K128K
KV cachef16q8_0q4_0quantizing the KV cache is a ~2× lever on the dominant term at long context
vision language 88text 874audio asr 38embedding 26audio tts 19video 5

What fits at 32K context

largest quantization that fits, per model · 1050 of 2118 indexed
ModelBest quantParamsWeights●KV●Total in memory◐Headroom◐tok/s◐
Fara1.5-9BIQ4_XS9.4B4.88 GiB0.53 GiB6.00 GiB0.00 GiB15±8.3%
QwenPaw-Flash-9BIQ4_XS9.4B4.88 GiB0.53 GiB6.00 GiB0.00 GiB15±8.3%
grug-9bIQ4_XS9.4B4.88 GiB0.53 GiB6.00 GiB0.00 GiB15±8.3%
OmniCoder-9BIQ4_XS9.4B4.88 GiB0.53 GiB6.00 GiB0.00 GiB15±8.3%
Qwen3.5-9B-NeoIQ4_XS9.7B4.88 GiB0.53 GiB6.00 GiB0.00 GiB15±8.3%
Assistant_Pepe_8BIQ3_XS—3.28 GiB2.13 GiB6.00 GiB0.00 GiB15±8.3%
Llama-3.2-3B-Instruct-abliteratedQ8_03.6B3.58 GiB1.86 GiB5.99 GiB0.01 GiB15±8.3%
Llama-3.2-3B-Instruct-uncensoredQ8_03.6B3.58 GiB1.86 GiB5.99 GiB0.01 GiB15±8.3%
Nemotron-Mini-4B-InstructQ5_K_L4.2B3.30 GiB2.13 GiB5.99 GiB0.01 GiB15±8.3%
dolphin-2.9.3-mistral-7B-32kI1-Q3_K_M7.2B3.28 GiB2.13 GiB5.99 GiB0.01 GiB15±8.3%
Mistral-7B-v0.3Q3_K_M7.2B3.28 GiB2.13 GiB5.99 GiB0.01 GiB15±8.3%
Mistral-7B-Instruct-v0.3-ParasiteI1-Q3_K_M7.2B3.28 GiB2.13 GiB5.99 GiB0.01 GiB15±8.3%
Mistral-7B-Instruct-v0.3-JbliteratedI1-Q3_K_M7.2B3.28 GiB2.13 GiB5.99 GiB0.01 GiB15±8.3%
Mistral-7B-Instruct-v0.3Q3_K_M7.2B3.28 GiB2.13 GiB5.99 GiB0.01 GiB15±8.3%
Mistral-7B-v0.3-Chinese-ChatQ3_K_M7.2B3.28 GiB2.13 GiB5.99 GiB0.01 GiB15±8.3%
mistral-7b-v0.3-bnb-4bitQ3_K_M7.5B3.28 GiB2.13 GiB5.99 GiB0.01 GiB15±8.3%
Mathstral-7B-v0.1Q3_K_M7.2B3.28 GiB2.13 GiB5.99 GiB0.01 GiB15±8.3%
Foundation-Sec-8B-InstructI1-IQ3_XS8.0B3.28 GiB2.13 GiB5.99 GiB0.01 GiB15±8.3%
Foundation-Sec-8B-Instruct-hereticI1-IQ3_XS8.0B3.28 GiB2.13 GiB5.99 GiB0.01 GiB15±8.3%
Meta-Llama-3-8B-InstructIQ3_XS8.0B3.28 GiB2.13 GiB5.99 GiB0.01 GiB15±8.3%
MiniCPM-Llama3-V-2_5IQ3_XS8.5B3.28 GiB2.13 GiB5.99 GiB0.01 GiB15±8.3%
Meta-Llama-3-8BIQ3_XS8.0B3.28 GiB2.13 GiB5.99 GiB0.01 GiB15±8.3%
Llama-3.1-Tulu-3-8BIQ3_XS8.0B3.28 GiB2.13 GiB5.99 GiB0.01 GiB15±8.3%
Llama-3-Groq-8B-Tool-UseIQ3_XS8.0B3.28 GiB2.13 GiB5.99 GiB0.01 GiB15±8.3%
llama3.1-heretic-unsensoredI1-IQ3_XS8.0B3.28 GiB2.13 GiB5.99 GiB0.01 GiB15±8.3%
Dolphin3.0-Llama3.1-8BIQ3_XS8.0B3.28 GiB2.13 GiB5.99 GiB0.01 GiB15±8.3%
dolphin-2.9.4-llama3.1-8bIQ3_XS8.0B3.28 GiB2.13 GiB5.99 GiB0.01 GiB15±8.3%
dolphin-2.9-llama3-8bIQ3_XS8.0B3.28 GiB2.13 GiB5.99 GiB0.01 GiB15±8.3%
Anubis-Mini-8B-v1I1-IQ3_XS8.0B3.28 GiB2.13 GiB5.99 GiB0.01 GiB15±8.3%
LLAMA-3_8B_Unaligned_BETAIQ3_XS8.0B3.28 GiB2.13 GiB5.99 GiB0.01 GiB15±8.3%
Meta-Llama-3-8BIQ3_XS8.0B3.28 GiB2.13 GiB5.99 GiB0.01 GiB15±8.3%
Llama3-ChatQA-1.5-8BIQ3_XS8.0B3.28 GiB2.13 GiB5.99 GiB0.01 GiB15±8.3%
L3.1-Dark-Reasoning-LewdPlay-evo-Hermes-R1-Uncensored-8B-hereticI1-IQ3_XS8.0B3.28 GiB2.13 GiB5.99 GiB0.01 GiB15±8.3%
L3.1-Dark-Reasoning-LewdPlay-evo-Hermes-R1-Uncensored-8BI1-IQ3_XS8.0B3.28 GiB2.13 GiB5.99 GiB0.01 GiB15±8.3%
Gluon-8BI1-IQ3_XS8.0B3.28 GiB2.13 GiB5.99 GiB0.01 GiB15±8.3%
Llama3.3-8B-Instruct-Thinking-Heretic-Uncensored-Claude-4.5-Opus-High-ReasoningI1-IQ3_XS8.0B3.28 GiB2.13 GiB5.99 GiB0.01 GiB15±8.3%
Llama3.3-8B-Instruct-Thinking-Claude-4.5-Opus-High-ReasoningI1-IQ3_XS8.0B3.28 GiB2.13 GiB5.99 GiB0.01 GiB15±8.3%
Hypnos-i1-8BIQ3_XS8.0B3.28 GiB2.13 GiB5.99 GiB0.01 GiB15±8.3%
grok-oss-Apollyon-8B-hereticI1-IQ3_XS8.0B3.28 GiB2.13 GiB5.99 GiB0.01 GiB15±8.3%
Llama-3.3-8B-Instruct-128K-JbliteratedI1-IQ3_XS8.0B3.28 GiB2.13 GiB5.99 GiB0.01 GiB15±8.3%
PsyCoPref-Llama3-8BI1-IQ3_XS8.0B3.28 GiB2.13 GiB5.99 GiB0.01 GiB15±8.3%
Llama3.1-GptDeluxe-8BI1-IQ3_XS8.0B3.28 GiB2.13 GiB5.99 GiB0.01 GiB15±8.3%
SelfCite-8B-CC-SFTI1-IQ3_XS8.0B3.28 GiB2.13 GiB5.99 GiB0.01 GiB15±8.3%
ai-girlfriend-v2I1-IQ3_XS8.0B3.28 GiB2.13 GiB5.99 GiB0.01 GiB15±8.3%
Llama-3.3-8B-Instruct-128K_AbliteratedI1-IQ3_XS8.0B3.28 GiB2.13 GiB5.99 GiB0.01 GiB15±8.3%
Aisha-Uncensored-v2I1-IQ3_XS8.0B3.28 GiB2.13 GiB5.99 GiB0.01 GiB15±8.3%
grok-oss-Apollyon-8BI1-IQ3_XS8.0B3.28 GiB2.13 GiB5.99 GiB0.01 GiB15±8.3%
HARC-Llama-3.1-8B-InstructI1-IQ3_XS8.0B3.28 GiB2.13 GiB5.99 GiB0.01 GiB15±8.3%
Lumen-1.2.5I1-IQ3_XS8.0B3.28 GiB2.13 GiB5.99 GiB0.01 GiB15±8.3%
Llama-3.3-8B-Instruct-128KI1-IQ3_XS8.0B3.28 GiB2.13 GiB5.99 GiB0.01 GiB15±8.3%
Stoicism1_Llama3.1-8b-instructI1-IQ3_XS8.0B3.28 GiB2.13 GiB5.99 GiB0.01 GiB15±8.3%
grok-oss-Thanatos-8BI1-IQ3_XS8.0B3.28 GiB2.13 GiB5.99 GiB0.01 GiB15±8.3%
Llama-3.1-8B-InstructIQ3_XS8.0B3.28 GiB2.13 GiB5.99 GiB0.01 GiB15±8.3%
Llama-3.1-8B-Lexi-Uncensored-V2IQ3_XS8.0B3.28 GiB2.13 GiB5.99 GiB0.01 GiB15±8.3%
OpenElla-NovelWriter-Requiem-AscendedI1-IQ3_XS8.0B3.28 GiB2.13 GiB5.99 GiB0.01 GiB15±8.3%
Llama-3.1-Nemotron-Nano-8B-v1IQ3_XS8.0B3.28 GiB2.13 GiB5.99 GiB0.01 GiB15±8.3%
llama-joycaption-beta-one-hf-llavaI1-IQ3_XS8.5B3.28 GiB2.13 GiB5.99 GiB0.01 GiB15±8.3%
DeepSeek-R1-Distill-Llama-8BIQ3_XS8.0B3.28 GiB2.13 GiB5.99 GiB0.01 GiB15±8.3%
Llama-3.1-Swallow-8B-Instruct-v0.5IQ3_XS8.0B3.28 GiB2.13 GiB5.99 GiB0.01 GiB15±8.3%
DarkIdol-Llama-3.1-8B-Instruct-1.3-UncensoredI1-IQ3_XS8.0B3.28 GiB2.13 GiB5.99 GiB0.01 GiB15±8.3%
●From the file◐Predictedwhat these mean

Speed is modeled, not measured: decode is memory-bandwidth bound, so tokens per second is bytes read per token against achievable bandwidth. Mixture-of-experts models carry a wider band because only the routed experts are read each step, and few have been measured publicly.

Questions people ask

What AI models can a Apple M3 run?
1050 of 2118 indexed open-weight models fit a Apple M3 at 32,768 context with q8_0 KV cache, the largest being Fara1.5-9B at IQ4_XS. That covers text, vision-language, image, video and speech models.
How much usable memory does a Apple M3 actually have?
Its nameplate is 8 GB, but about 5.58 GiB is available to a model once driver and compositor overhead is accounted for, and only 6 GB of the pool can be allocated to the GPU at all.
Is a Apple M3 fast for local AI?
Its memory bandwidth is 102 GB/s, and that figure — not teraflops — is what governs token generation speed. Capacity decides what you can run; bandwidth decides how fast it runs.