How much VRAM does Llama-2-7b-hf need?

Q4_K_M is exactly 4,081,004,224 bytes (3.80 GiB) in weights. Add the KV cache, which depends on context length, plus roughly half a gigabyte of runtime overhead.

How large is Llama-2-7b-hf's KV cache?

16.00 GiB at 32K context with an f16 cache, computed per layer from the model's architecture. Quantizing the cache to q8_0 roughly halves it.

meta-llama · text

Llama-2-7b-hf

meta-llama/Llama-2-7b-hf

Llama-2-7b-hf at Q4_K_M is exactly 4,081,004,224 bytes (3.80 GiB / 4.08 GB) — an effective 4.845 bits per weight, not the nominal 4. Its KV cache at 32K is 16.00 GiB.

From the file· summed from 1 file(s)From the file· KV from mirror (mirror:NousResearch/Llama-2-7b-hf)

Parameters

6.7B

Architecture

llama

32 layers

Context

4,096

native (config.json)

License

llama2

Shipped quantizations

● exact bytes, summed from published files

Quant	Size●	Exact bytes●	Effective bpw●	Tensors●	Publisher
Q2_K	2.63 GiB	2,825,940,672	3.355	—	TheBloke
Q3_K_S	2.75 GiB	2,948,304,576	3.500	—	TheBloke
Q3_K_M	3.07 GiB	3,298,004,672	3.916	—	TheBloke
Q3_K_L	3.35 GiB	3,597,110,976	4.271	—	TheBloke
Q4_0	3.56 GiB	3,825,807,040	4.542	—	TheBloke
Q4_K_S	3.59 GiB	3,856,740,032	4.579	—	TheBloke
Q4_K_M	3.80 GiB	4,081,004,224	4.845	—	TheBloke
Q5_K_S	4.33 GiB	4,651,691,712	5.523	—	TheBloke
Q5_0	4.33 GiB	4,651,691,712	5.523	—	TheBloke
Q5_K_M	4.45 GiB	4,783,156,928	5.679	—	TheBloke
Q6_K	5.15 GiB	5,529,194,176	6.564	—	TheBloke
Q8_0	6.67 GiB	7,161,089,728	8.502	—	TheBloke

KV cache by context

computed per layer

Context	KV cache (f16)●	Flat formula	Overstated by	Full / windowed / recurrent
4,096	2.00 GiB	2.00 GiB	—	32 / 0 / 0
8,192	4.00 GiB	4.00 GiB	—	32 / 0 / 0
16,384	8.00 GiB	8.00 GiB	—	32 / 0 / 0
32,768	16.00 GiB	16.00 GiB	—	32 / 0 / 0
65,536	32.00 GiB	32.00 GiB	—	32 / 0 / 0
131,072	64.00 GiB	64.00 GiB	—	32 / 0 / 0

Compare with

same modality, comparable size

vs gemma-4-12B-it-qat-q4_0-unquantized vs gemma-4-E4B-it vs Qwen3-4B vs Qwen3-8B vs HyperCLOVAX-SEED-Text-Instruct-1.5B vs Llama-3.2-1B-Instruct All alternatives →Will it fit on my card? →

Why other calculators give a different number

A parameters × bits ÷ 8 estimate puts Q4_K_M at roughly 3.53 GiB. The real file is 3.80 GiB, because a quantization is a mixture and some tensors are always kept at higher precision.

Architecture

from mirror:NousResearch/Llama-2-7b-hf

Layers

Attention heads

KV heads

Head dim

128

Hidden size

4096

Vocab

32,000

Sliding window

none

SWA period

—

MLA

Experts

—

Experts per token

—

use_sliding_window

—

Questions people ask

How much VRAM does Llama-2-7b-hf need?: Q4_K_M is exactly 4,081,004,224 bytes (3.80 GiB) in weights. Add the KV cache, which depends on your context length, plus roughly half a gigabyte of runtime overhead.
How large is Llama-2-7b-hf's KV cache?: 16.00 GiB at 32K context with an f16 cache, computed per layer. Quantizing the cache to q8_0 roughly halves it, which is often the difference between a context length fitting and not.
Which quantization of Llama-2-7b-hf should I use?: Q4_K_M is the usual default. Pick the largest quantization that fits your card at the context you actually need — the table above gives exact sizes for every one published.