KV cache
Stored attention keys and values for tokens already processed, so they aren't recomputed each step.
During generation a transformer attends over every previous token. Rather than recompute their key and value projections at each step, they are cached. That cache grows with context length and, past roughly 32K tokens, is often larger than the model weights themselves.
Its size is a per-layer sum, not one formula applied uniformly. Layers using sliding-window attention cache only a fixed window; models using latent attention allocate no value cache at all; hybrid models mix in linear-attention layers whose state does not grow with context. Any calculator multiplying context by layers by heads gets all three cases wrong.