How much VRAM does granite-4.1-3b need?

Q4_K_M is exactly 2,099,501,664 bytes (1.96 GiB) in weights. Add the KV cache, which depends on context length, plus roughly half a gigabyte of runtime overhead.

How large is granite-4.1-3b's KV cache?

2.50 GiB at 32K context with an f16 cache, computed per layer from the model's architecture. Quantizing the cache to q8_0 roughly halves it.

ibm-granite · text

granite-4.1-3b

ibm-granite/granite-4.1-3b

granite-4.1-3b at Q4_K_M is exactly 2,099,501,664 bytes (1.96 GiB / 2.10 GB) — an effective 4.936 bits per weight, not the nominal 4. Its KV cache at 32K is 2.50 GiB.

From the file· summed from 1 file(s)From the file· KV per layer

Parameters

3.4B

Architecture

granite

40 layers

Context

131,072

native (config.json)

License

apache-2.0

Shipped quantizations

● exact bytes, summed from published files

Quant	Size●	Exact bytes●	Effective bpw●	Tensors●	Publisher
UD-IQ2_M	1.20 GiB	1,285,053,760	3.021	—	unsloth
Q2_K	1.28 GiB	1,371,437,664	3.224	—	ibm-granite
UD-IQ3_XXS	1.32 GiB	1,416,576,320	3.330	—	unsloth
Q3_K_S	1.46 GiB	1,566,816,864	3.684	—	ibm-granite
Q3_K_S	1.46 GiB	1,566,817,600	3.684	—	unsloth
Q3_K_M	1.61 GiB	1,725,577,824	4.057	362	ibm-granite
Q3_K_M	1.61 GiB	1,725,578,560	4.057	362	unsloth
Q3_K_L	1.74 GiB	1,864,186,464	4.383	—	ibm-granite
IQ4_XS	1.76 GiB	1,894,497,600	4.454	362	unsloth
Q4_0	1.85 GiB	1,984,608,864	4.666	362	ibm-granite
IQ4_NL	1.85 GiB	1,991,163,200	4.681	—	unsloth
Q4_0	1.85 GiB	1,991,163,200	4.681	362	unsloth
Q4_K_S	1.86 GiB	1,998,371,424	4.698	—	ibm-granite
Q4_K_S	1.86 GiB	1,998,372,160	4.698	—	unsloth
Q4_K_M	1.96 GiB	2,099,501,664	4.936	362	ibm-granite
Q4_K_M	1.96 GiB	2,099,502,400	4.936	362	unsloth
Q4_1	2.03 GiB	2,181,216,864	5.128	—	ibm-granite
Q4_1	2.03 GiB	2,181,217,600	5.128	—	unsloth
Q5_K_S	2.21 GiB	2,377,824,864	5.590	—	ibm-granite
Q5_0	2.21 GiB	2,377,824,864	5.590	—	ibm-granite
Q5_K_S	2.21 GiB	2,377,825,600	5.590	—	unsloth
Q5_K_M	2.27 GiB	2,437,012,064	5.729	362	ibm-granite
Q5_K_M	2.27 GiB	2,437,012,800	5.729	362	unsloth
Q5_1	2.40 GiB	2,574,432,864	6.052	—	ibm-granite
Q6_K	2.60 GiB	2,795,616,864	6.572	362	ibm-granite
Q6_K	2.60 GiB	2,795,617,600	6.572	362	unsloth
Q8_0	3.37 GiB	3,619,691,104	8.510	362	ibm-granite
Q8_0	3.37 GiB	3,619,691,840	8.510	362	unsloth
BF16	6.34 GiB	6,809,655,904	16.009	—	ibm-granite
BF16	6.34 GiB	6,809,656,384	16.009	—	unsloth

KV cache by context

computed per layer

Context	KV cache (f16)●	Flat formula	Overstated by	Full / windowed / recurrent
4,096	0.31 GiB	0.31 GiB	—	40 / 0 / 0
8,192	0.63 GiB	0.63 GiB	—	40 / 0 / 0
16,384	1.25 GiB	1.25 GiB	—	40 / 0 / 0
32,768	2.50 GiB	2.50 GiB	—	40 / 0 / 0
65,536	5.00 GiB	5.00 GiB	—	40 / 0 / 0
131,072	10.00 GiB	10.00 GiB	—	40 / 0 / 0

Compare with

same modality, comparable size

vs Qwen3-4B vs Qwen3-8B vs HyperCLOVAX-SEED-Text-Instruct-1.5B vs Llama-3.2-1B-Instruct vs Qwen3-VL-8B-Instruct-abliterated-v1 vs Llama-3.1-8B-Instruct All alternatives →Will it fit on my card? →

Why other calculators give a different number

A parameters × bits ÷ 8 estimate puts Q4_K_M at roughly 1.78 GiB. The real file is 1.96 GiB, because a quantization is a mixture and some tensors are always kept at higher precision.

Architecture

from config.json

Layers

Attention heads

KV heads

Head dim

Hidden size

2560

Vocab

100,352

Sliding window

none

SWA period

—

MLA

Experts

—

Experts per token

—

use_sliding_window

—

Questions people ask

How much VRAM does granite-4.1-3b need?: Q4_K_M is exactly 2,099,501,664 bytes (1.96 GiB) in weights. Add the KV cache, which depends on your context length, plus roughly half a gigabyte of runtime overhead.
How large is granite-4.1-3b's KV cache?: 2.50 GiB at 32K context with an f16 cache, computed per layer. Quantizing the cache to q8_0 roughly halves it, which is often the difference between a context length fitting and not.
Which quantization of granite-4.1-3b should I use?: Q4_K_M is the usual default. Pick the largest quantization that fits your card at the context you actually need — the table above gives exact sizes for every one published.