How much VRAM does nomic-embed-text-v2-moe need?

Q4_K_M is exactly 344,120,288 bytes (0.32 GiB) in weights. Add the KV cache, which depends on context length, plus roughly half a gigabyte of runtime overhead.

Is nomic-embed-text-v2-moe a mixture-of-experts model?

Yes — 8 experts with null routed per token. Every expert must be resident in memory, but only the routed ones are read per token, so its memory requirement and its speed behave very differently.

nomic-ai · embedding · mixture of experts

nomic-embed-text-v2-moe

nomic-ai/nomic-embed-text-v2-moe

nomic-embed-text-v2-moe at Q4_K_M is exactly 344,120,288 bytes (0.32 GiB / 0.34 GB) — an effective 5.792 bits per weight, not the nominal 4.

From the file· summed from 1 file(s)From the file· KV per layer

Parameters

475M

total, not active

Architecture

nomic-bert-moe

12 layers

Context

—

native (config.json)

License

apache-2.0

Shipped quantizations

● exact bytes, summed from published files

Quant	Size●	Exact bytes●	Effective bpw●	Tensors●	Publisher
Q2_K	0.25 GiB	273,286,112	4.600	—	nomic-ai
Q3_K_S	0.27 GiB	288,381,920	4.854	—	nomic-ai
Q3_K_M	0.29 GiB	308,122,592	5.186	—	nomic-ai
Q3_K_L	0.30 GiB	322,223,072	5.424	—	nomic-ai
Q4_0	0.30 GiB	324,158,432	5.456	—	nomic-ai
Q4_K_S	0.30 GiB	325,338,080	5.476	—	nomic-ai
Q4_1	0.32 GiB	341,853,152	5.754	—	nomic-ai
Q4_K_M	0.32 GiB	344,120,288	5.792	—	nomic-ai
Q5_K_S	0.33 GiB	359,547,872	6.052	—	nomic-ai
IQ4_XS	0.34 GiB	360,472,608	6.067	—	cstr
Q4_K	0.34 GiB	369,321,120	6.216	—	cstr
Q5_K_M	0.35 GiB	370,828,256	6.242	—	nomic-ai
Q6_K	0.37 GiB	397,149,152	6.685	—	nomic-ai
Q8_0	0.48 GiB	510,897,312	8.599	—	cstr
Q8_0	0.48 GiB	512,225,120	8.622	—	nomic-ai
Q8_0	0.48 GiB	512,225,120	8.622	—	ggml-org
F16	0.89 GiB	957,680,480	16.119	—	nomic-ai
BF16	0.89 GiB	957,680,480	16.119	—	nomic-ai
F16	1.31 GiB	1,409,372,064	23.722	—	cstr
F32	1.78 GiB	1,907,985,280	32.115	—	nomic-ai

No KV cache

architectural, not a gap in our data

This architecture allocates no KV cache. Encoder and embedding models process their input in one pass rather than generating token by token, so there is nothing to carry forward between steps and memory does not grow with context. Its footprint is the weights plus a working buffer, and that is the whole story.

Compare with

same modality, comparable size

vs jina-embeddings-v5-text-small vs embeddinggemma-300m-qat-q8_0-unquantized vs KaLM-embedding-multilingual-mini-instruct-v2.5 vs nomic-embed-text-v1.5 vs jina-embeddings-v5-text-nano vs bge-m3 All alternatives →Will it fit on my card? →

Why other calculators give a different number

A parameters × bits ÷ 8 estimate puts Q4_K_M at roughly 0.25 GiB. The real file is 0.32 GiB, because a quantization is a mixture and some tensors are always kept at higher precision.

Architecture

from config.json

Layers

Attention heads

—

KV heads

—

Head dim

—

Hidden size

—

Vocab

250,048

Sliding window

none

SWA period

—

MLA

Experts

Experts per token

—

use_sliding_window

—

Questions people ask

How much VRAM does nomic-embed-text-v2-moe need?: Q4_K_M is exactly 344,120,288 bytes (0.32 GiB) in weights. Add the KV cache, which depends on your context length, plus roughly half a gigabyte of runtime overhead.
Is nomic-embed-text-v2-moe a mixture-of-experts model?: Yes — 8 experts, null routed per token. Every expert must be resident, but only the routed ones are read per token, which is why its memory requirement and its speed behave very differently.
Which quantization of nomic-embed-text-v2-moe should I use?: Q4_K_M is the usual default. Pick the largest quantization that fits your card at the context you actually need — the table above gives exact sizes for every one published.