How much VRAM does OpenELM-3B-Instruct need?

Q4_K_M is exactly 1,887,788,096 bytes (1.76 GiB) in weights. Add the KV cache, which depends on context length, plus roughly half a gigabyte of runtime overhead.

apple · text

OpenELM-3B-Instruct

apple/OpenELM-3B-Instruct

OpenELM-3B-Instruct at Q4_K_M is exactly 1,887,788,096 bytes (1.76 GiB / 1.89 GB) — an effective 4.973 bits per weight, not the nominal 4.

From the file· summed from 1 file(s)From the file· KV per layer

Parameters

3.0B

Architecture

openelm

null layers

Context

—

native (config.json)

License

apple-amlr

Shipped quantizations

● exact bytes, summed from published files

Quant	Size●	Exact bytes●	Effective bpw●	Tensors●	Publisher
Q2_K	1.07 GiB	1,150,219,328	3.030	—	mradermacher
IQ3_XS	1.20 GiB	1,291,703,360	3.403	—	mradermacher
IQ3_S	1.25 GiB	1,344,787,520	3.543	—	mradermacher
Q3_K_S	1.25 GiB	1,344,787,520	3.543	—	mradermacher
IQ3_M	1.34 GiB	1,435,657,280	3.782	—	mradermacher
Q3_K_M	1.43 GiB	1,533,236,288	4.039	—	mradermacher
IQ4_XS	1.54 GiB	1,649,333,312	4.345	—	mradermacher
Q3_K_L	1.55 GiB	1,659,753,536	4.373	—	mradermacher
Q4_K_S	1.62 GiB	1,739,920,448	4.584	—	mradermacher
Q4_K_M	1.76 GiB	1,887,788,096	4.973	—	mradermacher
Q5_K_S	1.96 GiB	2,102,268,992	5.538	—	mradermacher
Q5_K_M	2.06 GiB	2,207,448,128	5.816	—	mradermacher
Q6_K	2.32 GiB	2,492,486,720	6.566	—	mradermacher
Q8_0	3.01 GiB	3,227,868,224	8.504	—	mradermacher
F16	5.66 GiB	6,074,506,304	16.003	—	mradermacher

Compare with

same modality, comparable size

vs Qwen3-4B vs Qwen3-8B vs HyperCLOVAX-SEED-Text-Instruct-1.5B vs Llama-3.2-1B-Instruct vs Qwen3-VL-8B-Instruct-abliterated-v1 vs Llama-3.1-8B-Instruct All alternatives →Will it fit on my card? →

Why other calculators give a different number

A parameters × bits ÷ 8 estimate puts Q4_K_M at roughly 1.59 GiB. The real file is 1.76 GiB, because a quantization is a mixture and some tensors are always kept at higher precision.

Architecture

from config.json

Layers

—

Attention heads

—

KV heads

—

Head dim

128

Hidden size

—

Vocab

32,000

Sliding window

none

SWA period

—

MLA

Experts

—

Experts per token

—

use_sliding_window

—

Questions people ask

How much VRAM does OpenELM-3B-Instruct need?: Q4_K_M is exactly 1,887,788,096 bytes (1.76 GiB) in weights. Add the KV cache, which depends on your context length, plus roughly half a gigabyte of runtime overhead.
Which quantization of OpenELM-3B-Instruct should I use?: Q4_K_M is the usual default. Pick the largest quantization that fits your card at the context you actually need — the table above gives exact sizes for every one published.