How much VRAM does dots.llm1.inst need?

Q4_K_M is exactly 94,484,008,992 bytes (88.00 GiB) in weights. Add the KV cache, which depends on context length, plus roughly half a gigabyte of runtime overhead.

How large is dots.llm1.inst's KV cache?

31.00 GiB at 32K context with an f16 cache, computed per layer from the model's architecture. Quantizing the cache to q8_0 roughly halves it.

Is dots.llm1.inst a mixture-of-experts model?

Yes — 128 experts with 6 routed per token. Every expert must be resident in memory, but only the routed ones are read per token, so its memory requirement and its speed behave very differently.

rednote-hilab · text · mixture of experts

dots.llm1.inst

rednote-hilab/dots.llm1.inst

dots.llm1.inst at Q4_K_M is exactly 94,484,008,992 bytes (88.00 GiB / 94.48 GB) — an effective 5.294 bits per weight, not the nominal 4. Its KV cache at 32K is 31.00 GiB.

From the file· summed from 2 file(s)From the file· KV per layer

Parameters

143B

total, not active

Architecture

dots1

62 layers

Context

32,768

native (config.json)

License

mit

Shipped quantizations

● exact bytes, summed from published files

Quant	Size●	Exact bytes●	Effective bpw●	Tensors●	Publisher
IQ1_S	44.04 GiB	47,289,962,176	2.650	—	bartowski
UD-TQ1_0	44.71 GiB	48,010,350,432	2.690	—	unsloth
IQ1_M	44.84 GiB	48,146,886,336	2.698	—	bartowski
UD-IQ1_S	44.94 GiB	48,252,571,488	2.704	—	unsloth
UD-IQ1_M2 shards	47.01 GiB	50,477,936,672	2.828	—	unsloth
IQ2_XXS2 shards	47.03 GiB	50,501,685,152	2.830	—	bartowski
IQ2_XS2 shards	50.22 GiB	53,924,204,416	3.022	—	bartowski
UD-IQ2_XXS2 shards	50.31 GiB	54,023,901,216	3.027	—	unsloth
IQ2_S2 shards	50.40 GiB	54,117,486,464	3.032	—	bartowski
UD-IQ2_M2 shards	52.59 GiB	56,469,180,448	3.164	—	unsloth
IQ2_M2 shards	53.68 GiB	57,643,814,784	3.230	—	bartowski
Q2_K2 shards	54.13 GiB	58,126,970,912	3.257	—	unsloth
Q2_K_L2 shards	54.27 GiB	58,272,952,352	3.265	—	unsloth
Q2_K2 shards	56.10 GiB	60,235,419,552	3.375	—	bartowski
Q2_K_L2 shards	56.67 GiB	60,843,675,520	3.409	—	bartowski
IQ3_XXS2 shards	60.62 GiB	65,089,134,496	3.647	—	bartowski
UD-IQ3_XXS2 shards	60.94 GiB	65,437,970,464	3.667	—	unsloth
IQ3_XS2 shards	61.71 GiB	66,258,714,496	3.713	—	bartowski
Q3_K_S2 shards	63.05 GiB	67,700,269,088	3.793	—	unsloth
Q3_K_S2 shards	64.78 GiB	69,560,156,032	3.898	—	bartowski
IQ3_M2 shards	67.49 GiB	72,472,149,888	4.061	—	bartowski
Q3_K_M2 shards	67.50 GiB	72,479,145,856	4.061	—	bartowski
Q3_K_L2 shards	68.66 GiB	73,720,921,984	4.131	—	bartowski
Q3_K_M2 shards	68.74 GiB	73,806,528,544	4.136	—	unsloth
IQ4_XS2 shards	72.25 GiB	77,575,778,336	4.347	—	unsloth
IQ4_XS2 shards	73.57 GiB	78,990,536,576	4.426	—	bartowski
IQ4_NL2 shards	75.05 GiB	80,588,943,392	4.516	—	unsloth
Q4_02 shards	75.32 GiB	80,872,894,496	4.532	—	unsloth
IQ4_NL3 shards	76.27 GiB	81,892,192,256	4.589	—	bartowski
Q4_03 shards	77.44 GiB	83,149,090,816	4.659	—	bartowski
Q4_K_S2 shards	80.58 GiB	86,519,967,776	4.848	—	unsloth
Q4_K_S3 shards	82.81 GiB	88,921,862,144	4.982	—	bartowski
Q4_12 shards	83.33 GiB	89,471,381,536	5.013	—	unsloth
Q4_13 shards	84.24 GiB	90,447,671,296	5.068	—	bartowski
Q4_K_M2 shards	88.00 GiB	94,484,008,992	5.294	—	unsloth
Q4_K_M3 shards	89.52 GiB	96,120,311,808	5.386	—	bartowski
Q4_K_L3 shards	89.95 GiB	96,582,586,368	5.412	—	bartowski
Q5_K_S3 shards	94.22 GiB	101,170,999,424	5.669	—	unsloth
Q5_K_S3 shards	95.08 GiB	102,092,820,448	5.721	—	bartowski
Q5_K_M3 shards	100.57 GiB	107,986,194,592	6.051	—	unsloth

KV cache by context

computed per layer

Context	KV cache (f16)●	Flat formula	Overstated by	Full / windowed / recurrent
4,096	3.88 GiB	3.88 GiB	—	62 / 0 / 0
8,192	7.75 GiB	7.75 GiB	—	62 / 0 / 0
16,384	15.50 GiB	15.50 GiB	—	62 / 0 / 0
32,768	31.00 GiB	31.00 GiB	—	62 / 0 / 0
65,536	62.00 GiB	62.00 GiB	—	62 / 0 / 0
131,072	124.00 GiB	124.00 GiB	—	62 / 0 / 0

Compare with

same modality, comparable size

vs Qwen3-Coder-30B-A3B-Instruct vs Qwen3.6-27B vs Qwen3.8-27B vs DeepSeek-V4-Flash vs Qwen3-30B-A3B-Thinking-2507 vs Laguna-XS-2.1 All alternatives →Will it fit on my card? →

Why other calculators give a different number

A parameters × bits ÷ 8 estimate puts Q4_K_M at roughly 74.80 GiB. The real file is 88.00 GiB, because a quantization is a mixture and some tensors are always kept at higher precision.

Architecture

from config.json

Layers

Attention heads

KV heads

Head dim

128

Hidden size

4096

Vocab

152,064

Sliding window

none

SWA period

—

MLA

Experts

128

Experts per token

use_sliding_window

false

Questions people ask

How much VRAM does dots.llm1.inst need?: Q4_K_M is exactly 94,484,008,992 bytes (88.00 GiB) in weights. Add the KV cache, which depends on your context length, plus roughly half a gigabyte of runtime overhead.
How large is dots.llm1.inst's KV cache?: 31.00 GiB at 32K context with an f16 cache, computed per layer. Quantizing the cache to q8_0 roughly halves it, which is often the difference between a context length fitting and not.
Is dots.llm1.inst a mixture-of-experts model?: Yes — 128 experts, 6 routed per token. Every expert must be resident, but only the routed ones are read per token, which is why its memory requirement and its speed behave very differently.
Which quantization of dots.llm1.inst should I use?: Q4_K_M is the usual default. Pick the largest quantization that fits your card at the context you actually need — the table above gives exact sizes for every one published.