How much VRAM does gemma-2b need?

Q4_K_M is exactly 1,495,245,728 bytes (1.39 GiB) in weights. Add the KV cache, which depends on context length, plus roughly half a gigabyte of runtime overhead.

How large is gemma-2b's KV cache?

0.56 GiB at 32K context with an f16 cache, computed per layer from the model's architecture. Quantizing the cache to q8_0 roughly halves it.

google · text

gemma-2b

google/gemma-2b

gemma-2b at Q4_K_M is exactly 1,495,245,728 bytes (1.39 GiB / 1.50 GB) — an effective 4.773 bits per weight, not the nominal 4. Its KV cache at 32K is 0.56 GiB.

From the file· summed from 1 file(s)From the file· KV from mirror (mirror:unsloth/gemma-2b)

Parameters

2.5B

Architecture

gemma

18 layers

Context

8,192

native (config.json)

License

cc-by-nc-4.0

Shipped quantizations

● exact bytes, summed from published files

Quant	Size●	Exact bytes●	Effective bpw●	Tensors●	Publisher
Q2_K	0.84 GiB	899,965,856	2.873	—	brittlewis12
Q3_K_S	1.01 GiB	1,083,296,672	3.458	—	brittlewis12
Q2_K	1.08 GiB	1,157,923,968	3.696	—	MaziyarPanahi
Q2_K	1.08 GiB	1,157,962,304	3.696	—	NexaAI
Q3_K_M	1.10 GiB	1,179,118,496	3.764	—	brittlewis12
Q3_K_L	1.17 GiB	1,260,907,424	4.025	—	brittlewis12
Q3_K_S	1.20 GiB	1,287,980,160	4.111	—	MaziyarPanahi
Q3_K_S	1.20 GiB	1,288,018,496	4.112	—	NexaAI
Q3_K_M	1.29 GiB	1,383,801,984	4.417	—	MaziyarPanahi
Q3_K	1.29 GiB	1,383,840,320	4.417	—	NexaAI
Q3_K_M	1.29 GiB	1,383,840,320	4.417	—	NexaAI
Q4_K_S	1.33 GiB	1,424,823,200	4.548	—	brittlewis12
Q3_K_L	1.36 GiB	1,465,590,912	4.678	—	MaziyarPanahi
Q3_K_L	1.36 GiB	1,465,629,248	4.678	—	NexaAI
Q4_K_M	1.39 GiB	1,495,245,728	4.773	—	brittlewis12
Q4_0	1.44 GiB	1,551,227,456	4.952	—	NexaAI
Q4_K_S	1.45 GiB	1,559,839,872	4.979	—	MaziyarPanahi
Q4_K_S	1.45 GiB	1,559,878,208	4.979	—	NexaAI
Q4_K_M	1.52 GiB	1,630,262,400	5.204	—	MaziyarPanahi
Q4_K	1.52 GiB	1,630,300,736	5.204	—	NexaAI
Q4_K_M	1.52 GiB	1,630,300,736	5.204	—	NexaAI
Q4_1	1.56 GiB	1,675,090,496	5.347	—	NexaAI
Q5_K_S	1.61 GiB	1,729,467,296	5.521	—	brittlewis12
Q5_K_M	1.65 GiB	1,770,202,016	5.651	—	brittlewis12
Q5_K_S	1.68 GiB	1,798,915,200	5.742	—	MaziyarPanahi
Q5_K_S	1.68 GiB	1,798,953,536	5.742	—	NexaAI
Q5_0	1.68 GiB	1,798,953,536	5.742	—	NexaAI
Q5_K_M	1.71 GiB	1,839,649,920	5.872	—	MaziyarPanahi
Q5_K	1.71 GiB	1,839,688,256	5.872	—	NexaAI
Q5_K_M	1.71 GiB	1,839,688,256	5.872	—	NexaAI
Q5_1	1.79 GiB	1,922,816,576	6.138	—	NexaAI
Q6_K	1.92 GiB	2,062,124,160	6.582	—	MaziyarPanahi
Q6_K	1.92 GiB	2,062,162,496	6.583	—	NexaAI
Q6_K	1.92 GiB	2,062,343,072	6.583	—	brittlewis12
Q8_0	2.49 GiB	2,669,069,440	8.520	—	MaziyarPanahi
Q8_0	2.49 GiB	2,669,118,688	8.520	—	NexaAI
Q8_0	2.49 GiB	2,669,351,840	8.521	—	brittlewis12
F16	4.67 GiB	5,018,626,528	16.020	—	NexaAI

KV cache by context

computed per layer

Context	KV cache (f16)●	Flat formula	Overstated by	Full / windowed / recurrent
4,096	0.07 GiB	0.07 GiB	—	18 / 0 / 0
8,192	0.14 GiB	0.14 GiB	—	18 / 0 / 0
16,384	0.28 GiB	0.28 GiB	—	18 / 0 / 0
32,768	0.56 GiB	0.56 GiB	—	18 / 0 / 0
65,536	1.13 GiB	1.13 GiB	—	18 / 0 / 0
131,072	2.25 GiB	2.25 GiB	—	18 / 0 / 0

Compare with

same modality, comparable size

vs Qwen3-4B vs HyperCLOVAX-SEED-Text-Instruct-1.5B vs Llama-3.2-1B-Instruct vs Qwen3-VL-8B-Instruct-abliterated-v1 vs Llama-3.1-8B-Instruct vs gemma-3-1b-it All alternatives →Will it fit on my card? →

Why other calculators give a different number

A parameters × bits ÷ 8 estimate puts Q4_K_M at roughly 1.31 GiB. The real file is 1.39 GiB, because a quantization is a mixture and some tensors are always kept at higher precision.

Architecture

from mirror:unsloth/gemma-2b

Layers

Attention heads

KV heads

Head dim

256

Hidden size

2048

Vocab

256,000

Sliding window

none

SWA period

—

MLA

Experts

—

Experts per token

—

use_sliding_window

—

Questions people ask

How much VRAM does gemma-2b need?: Q4_K_M is exactly 1,495,245,728 bytes (1.39 GiB) in weights. Add the KV cache, which depends on your context length, plus roughly half a gigabyte of runtime overhead.
How large is gemma-2b's KV cache?: 0.56 GiB at 32K context with an f16 cache, computed per layer. Quantizing the cache to q8_0 roughly halves it, which is often the difference between a context length fitting and not.
Which quantization of gemma-2b should I use?: Q4_K_M is the usual default. Pick the largest quantization that fits your card at the context you actually need — the table above gives exact sizes for every one published.