How much VRAM does GLM-4.6V need?

Q4_K_M is exactly 70,448,211,840 bytes (65.61 GiB) in weights. Add the KV cache, which depends on context length, plus roughly half a gigabyte of runtime overhead.

How large is GLM-4.6V's KV cache?

5.75 GiB at 32K context with an f16 cache, computed per layer from the model's architecture. Quantizing the cache to q8_0 roughly halves it.

Is GLM-4.6V a mixture-of-experts model?

Yes — 128 experts with 8 routed per token. Every expert must be resident in memory, but only the routed ones are read per token, so its memory requirement and its speed behave very differently.

zai-org · vision language · mixture of experts

GLM-4.6V

zai-org/GLM-4.6V

GLM-4.6V at Q4_K_M is exactly 70,448,211,840 bytes (65.61 GiB / 70.45 GB) — an effective 5.232 bits per weight, not the nominal 4. Its KV cache at 32K is 5.75 GiB.

From the file· summed from 1 file(s)From the file· KV per layer

Parameters

108B

total, not active

Architecture

glm4moe

46 layers

Context

131,072

native (config.json)

License

mit

Shipped quantizations

● exact bytes, summed from published files

Quant	Size●	Exact bytes●	Effective bpw●	Tensors●	Publisher
IQ1_M	33.46 GiB	35,922,389,248	2.668	—	bartowski
UD-TQ1_0	34.27 GiB	36,795,788,000	2.733	—	unsloth
UD-IQ1_S	34.49 GiB	37,038,009,056	2.751	—	unsloth
IQ2_XXS	35.11 GiB	37,695,670,528	2.800	—	bartowski
UD-IQ1_M	35.93 GiB	38,576,794,336	2.865	—	unsloth
IQ2_XS	37.49 GiB	40,256,194,816	2.990	—	bartowski
IQ2_S	37.82 GiB	40,608,647,424	3.016	—	bartowski
UD-IQ2_XXS	38.37 GiB	41,197,513,440	3.060	—	unsloth
UD-IQ2_M	39.96 GiB	42,904,595,168	3.187	—	unsloth
IQ2_M	40.26 GiB	43,226,843,392	3.211	—	bartowski
Q2_K	40.84 GiB	43,855,039,200	3.257	—	unsloth
Q2_K_L	40.98 GiB	44,000,529,120	3.268	—	unsloth
Q2_K	41.64 GiB	44,712,216,832	3.321	—	bartowski
Q2_K_L	42.21 GiB	45,318,424,832	3.366	—	bartowski
IQ3_XXS	44.99 GiB	48,312,948,992	3.588	—	bartowski
IQ3_XS	45.85 GiB	49,236,359,424	3.657	—	bartowski
UD-IQ3_XXS	46.26 GiB	49,675,709,152	3.690	—	unsloth
Q3_K_S2 shards	47.41 GiB	50,903,890,848	3.781	—	unsloth
Q3_K_S2 shards	48.19 GiB	51,741,473,216	3.843	—	bartowski
IQ3_M2 shards	50.11 GiB	53,802,744,256	3.996	—	bartowski
Q3_K_M2 shards	50.11 GiB	53,806,594,496	3.996	—	bartowski
Q3_K_L2 shards	50.52 GiB	54,247,782,848	4.029	—	bartowski
Q3_K_M2 shards	51.64 GiB	55,444,962,208	4.118	—	unsloth
IQ4_XS2 shards	54.34 GiB	58,347,531,168	4.334	—	unsloth
IQ4_XS2 shards	54.80 GiB	58,846,095,808	4.371	—	bartowski
IQ4_NL2 shards	56.42 GiB	60,577,246,112	4.499	—	unsloth
Q4_02 shards	56.57 GiB	60,740,479,872	4.511	—	unsloth
IQ4_NL2 shards	56.83 GiB	61,016,828,384	4.532	—	bartowski
Q4_02 shards	57.48 GiB	61,721,127,392	4.584	—	bartowski
Q4_K_S2 shards	60.44 GiB	64,900,541,344	4.820	—	unsloth
Q4_K_S2 shards	61.63 GiB	66,170,595,776	4.915	—	bartowski
Q4_12 shards	62.53 GiB	67,142,396,832	4.987	—	unsloth
Q4_12 shards	62.66 GiB	67,280,513,504	4.997	—	bartowski
Q4_K_M	65.61 GiB	70,448,211,840	5.232	—	ggml-org
Q4_K_M2 shards	65.72 GiB	70,564,998,048	5.241	—	unsloth
Q4_K_M2 shards	66.46 GiB	71,358,089,664	5.300	—	bartowski
Q4_K_L2 shards	66.89 GiB	71,818,807,744	5.334	—	bartowski
Q5_K_S2 shards	70.67 GiB	75,879,582,624	5.636	—	unsloth
Q5_K_S2 shards	70.79 GiB	76,013,390,272	5.646	—	bartowski
Q5_K_M2 shards	75.20 GiB	80,750,267,296	5.998	—	unsloth

KV cache by context

computed per layer

Context	KV cache (f16)●	Flat formula	Overstated by	Full / windowed / recurrent
4,096	0.72 GiB	0.72 GiB	—	46 / 0 / 0
8,192	1.44 GiB	1.44 GiB	—	46 / 0 / 0
16,384	2.88 GiB	2.88 GiB	—	46 / 0 / 0
32,768	5.75 GiB	5.75 GiB	—	46 / 0 / 0
65,536	11.50 GiB	11.50 GiB	—	46 / 0 / 0
131,072	23.00 GiB	23.00 GiB	—	46 / 0 / 0

Compare with

same modality, comparable size

vs Qwen3.6-35B-A3B vs Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP vs Qwen3.5-9B vs gemma-4-26B-A4B-it vs gemma-4-12B-it vs Qwythos-9B-Claude-Mythos-5-1M All alternatives →Will it fit on my card? →

Why other calculators give a different number

A parameters × bits ÷ 8 estimate puts Q4_K_M at roughly 56.43 GiB. The real file is 65.61 GiB, because a quantization is a mixture and some tensors are always kept at higher precision.

Architecture

from config.json

Layers

Attention heads

KV heads

Head dim

128

Hidden size

4096

Vocab

151,552

Sliding window

none

SWA period

—

MLA

Experts

128

Experts per token

use_sliding_window

—

Questions people ask

How much VRAM does GLM-4.6V need?: Q4_K_M is exactly 70,448,211,840 bytes (65.61 GiB) in weights. Add the KV cache, which depends on your context length, plus roughly half a gigabyte of runtime overhead.
How large is GLM-4.6V's KV cache?: 5.75 GiB at 32K context with an f16 cache, computed per layer. Quantizing the cache to q8_0 roughly halves it, which is often the difference between a context length fitting and not.
Is GLM-4.6V a mixture-of-experts model?: Yes — 128 experts, 8 routed per token. Every expert must be resident, but only the routed ones are read per token, which is why its memory requirement and its speed behave very differently.
Which quantization of GLM-4.6V should I use?: Q4_K_M is the usual default. Pick the largest quantization that fits your card at the context you actually need — the table above gives exact sizes for every one published.