KV cache quantization
Storing the attention cache at reduced precision, roughly halving its size for little quality cost.
The cache can be stored at q8_0 (8.5 bits per element) or q4_0 (4.5 bits) instead of f16. At long context the cache is the dominant memory term, so this is often the difference between a context length fitting and not. q8_0 is widely considered close to free in quality terms; q4_0 is more aggressive.
It is probably the most useful setting that most local users never change.