How much VRAM does v6-Finch-7B-HF need?

Q4_K_M is exactly 4,776,703,776 bytes (4.45 GiB) in weights. Add the KV cache, which depends on context length, plus roughly half a gigabyte of runtime overhead.

How large is v6-Finch-7B-HF's KV cache?

16.00 GiB at 32K context with an f16 cache, computed per layer from the model's architecture. Quantizing the cache to q8_0 roughly halves it.

RWKV · text

v6-Finch-7B-HF

RWKV/v6-Finch-7B-HF

v6-Finch-7B-HF at Q4_K_M is exactly 4,776,703,776 bytes (4.45 GiB / 4.78 GB) — an effective 5.005 bits per weight, not the nominal 4. Its KV cache at 32K is 16.00 GiB.

From the file· summed from 1 file(s)From the file· KV per layer

Parameters

7.6B

Architecture

rwkv6

32 layers

Context

—

native (config.json)

License

apache-2.0

Shipped quantizations

● exact bytes, summed from published files

Quant	Size●	Exact bytes●	Effective bpw●	Tensors●	Publisher
IQ2_M	2.81 GiB	3,015,096,096	3.159	—	bartowski
Q2_K	2.87 GiB	3,078,010,656	3.225	—	bartowski
Q2_K_L	3.11 GiB	3,340,154,656	3.499	—	bartowski
Q3_K_L	3.55 GiB	3,814,111,008	3.996	—	bartowski
Q3_K_S	3.55 GiB	3,814,111,008	3.996	—	bartowski
IQ3_M	3.55 GiB	3,814,111,008	3.996	—	bartowski
IQ3_XS	3.55 GiB	3,814,111,008	3.996	—	bartowski
Q3_K_M	3.55 GiB	3,814,111,008	3.996	—	bartowski
IQ4_XS	4.24 GiB	4,550,211,360	4.767	—	bartowski
Q4_K_S	4.45 GiB	4,776,703,776	5.005	—	bartowski
Q4_K_M	4.45 GiB	4,776,703,776	5.005	—	bartowski
Q4_0	4.45 GiB	4,776,703,776	5.005	—	bartowski
Q4_K_L	4.63 GiB	4,975,933,216	5.213	—	bartowski
Q5_K_S	5.29 GiB	5,682,673,440	5.954	—	bartowski
Q5_K_M	5.29 GiB	5,682,673,440	5.954	—	bartowski
Q5_K_L	5.45 GiB	5,848,348,448	6.127	—	bartowski
Q6_K	6.19 GiB	6,645,266,208	6.962	—	bartowski
Q6_K_L	6.31 GiB	6,775,289,632	7.098	—	bartowski
Q8_0	7.88 GiB	8,465,594,144	8.869	—	bartowski
F16	14.45 GiB	15,512,024,608	16.252	—	bartowski

KV cache by context

computed per layer

Context	KV cache (f16)●	Flat formula	Overstated by	Full / windowed / recurrent
4,096	2.00 GiB	2.00 GiB	—	32 / 0 / 0
8,192	4.00 GiB	4.00 GiB	—	32 / 0 / 0
16,384	8.00 GiB	8.00 GiB	—	32 / 0 / 0
32,768	16.00 GiB	16.00 GiB	—	32 / 0 / 0
65,536	32.00 GiB	32.00 GiB	—	32 / 0 / 0
131,072	64.00 GiB	64.00 GiB	—	32 / 0 / 0

Compare with

same modality, comparable size

vs gemma-4-12B-it-qat-q4_0-unquantized vs gemma-4-E4B-it vs Qwen3-4B vs Qwen3-8B vs Llama-3.2-1B-Instruct vs llama-3-youko-8b All alternatives →Will it fit on my card? →

Why other calculators give a different number

A parameters × bits ÷ 8 estimate puts Q4_K_M at roughly 4.00 GiB. The real file is 4.45 GiB, because a quantization is a mixture and some tensors are always kept at higher precision.

Architecture

from config.json

Layers

Attention heads

KV heads

Head dim

Hidden size

4096

Vocab

65,536

Sliding window

none

SWA period

—

MLA

Experts

—

Experts per token

—

use_sliding_window

—

Questions people ask

How much VRAM does v6-Finch-7B-HF need?: Q4_K_M is exactly 4,776,703,776 bytes (4.45 GiB) in weights. Add the KV cache, which depends on your context length, plus roughly half a gigabyte of runtime overhead.
How large is v6-Finch-7B-HF's KV cache?: 16.00 GiB at 32K context with an f16 cache, computed per layer. Quantizing the cache to q8_0 roughly halves it, which is often the difference between a context length fitting and not.
Which quantization of v6-Finch-7B-HF should I use?: Q4_K_M is the usual default. Pick the largest quantization that fits your card at the context you actually need — the table above gives exact sizes for every one published.