How much VRAM does Qwen2.5-Omni-3B need?

Q4_K_M is exactly 2,104,931,648 bytes (1.96 GiB) in weights. Add the KV cache, which depends on context length, plus roughly half a gigabyte of runtime overhead.

Qwen · text

Qwen2.5-Omni-3B

Qwen/Qwen2.5-Omni-3B

Qwen2.5-Omni-3B at Q4_K_M is exactly 2,104,931,648 bytes (1.96 GiB / 2.10 GB) — an effective 3.041 bits per weight, not the nominal 4.

From the file· summed from 1 file(s)From the file· KV per layer

Parameters

5.5B

Architecture

qwen2vl

null layers

Context

—

native (config.json)

License

other

Shipped quantizations

● exact bytes, summed from published files

Quant	Size●	Exact bytes●	Effective bpw●	Tensors●	Publisher
UD-IQ1_S	0.89 GiB	960,038,944	1.387	—	unsloth
UD-IQ1_M	0.95 GiB	1,015,343,136	1.467	—	unsloth
UD-IQ2_XXS	1.03 GiB	1,104,390,176	1.596	—	unsloth
UD-IQ2_M	1.21 GiB	1,296,816,160	1.874	—	unsloth
Q2_K	1.28 GiB	1,376,856,096	1.989	—	unsloth
UD-IQ3_XXS	1.34 GiB	1,436,358,688	2.075	—	unsloth
Q2_K_L	1.35 GiB	1,449,785,376	2.095	—	unsloth
Q3_K_S	1.48 GiB	1,588,060,192	2.294	—	unsloth
Q3_K_M	1.61 GiB	1,724,178,464	2.491	—	unsloth
IQ4_XS	1.78 GiB	1,914,124,320	2.765	—	unsloth
IQ4_NL	1.86 GiB	2,000,238,624	2.890	—	unsloth
Q4_K_S	1.87 GiB	2,009,413,664	2.903	—	unsloth
Q4_K_M	1.96 GiB	2,104,931,648	3.041	—	ggml-org
Q4_K_M	1.96 GiB	2,104,932,384	3.041	—	unsloth
Q4_1	2.04 GiB	2,190,735,392	3.165	—	unsloth
Q5_K_S	2.22 GiB	2,383,591,456	3.444	—	unsloth
Q5_K_M	2.27 GiB	2,438,740,000	3.523	—	unsloth
Q6_K	2.60 GiB	2,793,410,592	4.036	—	unsloth
Q8_0	3.37 GiB	3,616,087,360	5.224	—	ggml-org
Q8_0	3.37 GiB	3,616,088,096	5.224	—	unsloth
F16	6.33 GiB	6,800,645,440	9.825	—	ggml-org
BF16	6.33 GiB	6,800,645,920	9.825	—	unsloth

Compare with

same modality, comparable size

vs Qwen3-4B vs Qwen3-8B vs HyperCLOVAX-SEED-Text-Instruct-1.5B vs Llama-3.2-1B-Instruct vs Qwen3-VL-8B-Instruct-abliterated-v1 vs Llama-3.1-8B-Instruct All alternatives →Will it fit on my card? →

Why other calculators give a different number

A parameters × bits ÷ 8 estimate puts Q4_K_M at roughly 2.90 GiB. The real file is 1.96 GiB, because a quantization is a mixture and some tensors are always kept at higher precision.

Architecture

from config.json

Layers

—

Attention heads

—

KV heads

—

Head dim

—

Hidden size

—

Vocab

—

Sliding window

none

SWA period

—

MLA

Experts

—

Experts per token

—

use_sliding_window

—

Questions people ask

How much VRAM does Qwen2.5-Omni-3B need?: Q4_K_M is exactly 2,104,931,648 bytes (1.96 GiB) in weights. Add the KV cache, which depends on your context length, plus roughly half a gigabyte of runtime overhead.
Which quantization of Qwen2.5-Omni-3B should I use?: Q4_K_M is the usual default. Pick the largest quantization that fits your card at the context you actually need — the table above gives exact sizes for every one published.