Can I run Kimi-K2-Instruct on a Apple M2 Ultra?
Not at these settings. No indexed quantization of Kimi-K2-Instruct fits Apple M2 Ultra at any context we compute, with q4_0 KV. The smallest shipped quantization is 226.87 GiB in weights alone, against 133.92 GiB usable. CPU offload can still run it, slowly.
Every quantization at every context
| Quant | Weights● | 4K◐ | 8K◐ | 16K◐ | 32K◐ | 64K◐ | 128K◐ |
|---|---|---|---|---|---|---|---|
| BF16 | 1912.15 GiB | 1912.9 | 1912.9 | 1913.1 | 1913.4 | 1914.0 | 1915.2 |
| Q8_0 | 1016.12 GiB | 1016.8 | 1016.9 | 1017.1 | 1017.4 | 1018.0 | 1019.2 |
| Q6_K | 784.76 GiB | 785.5 | 785.5 | 785.7 | 786.0 | 786.6 | 787.8 |
| Q5_K_M | 678.33 GiB | 679.0 | 679.1 | 679.3 | 679.6 | 680.2 | 681.4 |
| Q5_K_S | 658.04 GiB | 658.7 | 658.8 | 659.0 | 659.3 | 659.9 | 661.1 |
| Q4_1 | 598.40 GiB | 599.1 | 599.2 | 599.3 | 599.6 | 600.2 | 601.4 |
| Q4_K_M | 578.15 GiB | 578.9 | 578.9 | 579.1 | 579.4 | 580.0 | 581.2 |
| Q4_K_S | 542.73 GiB | 543.4 | 543.5 | 543.7 | 544.0 | 544.6 | 545.8 |
| Q4_0 | 540.74 GiB | 541.4 | 541.5 | 541.7 | 542.0 | 542.6 | 543.8 |
| IQ4_NL | 538.76 GiB | 539.5 | 539.5 | 539.7 | 540.0 | 540.6 | 541.8 |
| IQ4_XS | 508.98 GiB | 509.7 | 509.8 | 509.9 | 510.2 | 510.8 | 512.0 |
| Q3_K_M | 455.77 GiB | 456.5 | 456.5 | 456.7 | 457.0 | 457.6 | 458.8 |
| Q3_K_S | 412.03 GiB | 412.7 | 412.8 | 413.0 | 413.3 | 413.9 | 415.1 |
| UD-IQ3_XXS | 388.01 GiB | 388.7 | 388.8 | 388.9 | 389.2 | 389.8 | 391.1 |
| Q2_K_L | 347.81 GiB | 348.5 | 348.6 | 348.7 | 349.0 | 349.6 | 350.8 |
| Q2_K | 347.55 GiB | 348.3 | 348.3 | 348.5 | 348.8 | 349.4 | 350.6 |
| UD-IQ2_M | 323.27 GiB | 324.0 | 324.0 | 324.2 | 324.5 | 325.1 | 326.3 |
| UD-IQ2_XXS | 306.20 GiB | 306.9 | 307.0 | 307.1 | 307.4 | 308.0 | 309.2 |
| UD-IQ1_M | 283.34 GiB | 284.0 | 284.1 | 284.3 | 284.6 | 285.2 | 286.4 |
| UD-IQ1_S | 260.88 GiB | 261.6 | 261.7 | 261.8 | 262.1 | 262.7 | 263.9 |
| UD-TQ1_0 | 226.87 GiB | 227.6 | 227.7 | 227.8 | 228.1 | 228.7 | 229.9 |
Figures are GiB of total memory: weights plus KV cache plus compute buffer and backend overhead. Weights and KV are near-exact; the overhead term is modeled. Hover any cell for the breakdown.
Why other calculators disagree
A parameters × bits ÷ 8 estimate ignores two things that dominate at long context. First, the weights themselves are not the nominal rate — quantizations are mixtures, so the real file is consistently larger than the label implies. Second, this model uses latent attention and allocates no V cache at all, so any formula reading num_key_value_heads overstates its cache by more than an order of magnitude.