How much VRAM does parakeet-rnnt-0.6b need?

Q4_K_M is exactly 476,390,816 bytes (0.44 GiB) in weights. Add the KV cache, which depends on context length, plus roughly half a gigabyte of runtime overhead.

nvidia · audio asr

parakeet-rnnt-0.6b

nvidia/parakeet-rnnt-0.6b

parakeet-rnnt-0.6b at Q4_K_M is exactly 476,390,816 bytes (0.44 GiB / 0.48 GB) — an effective 6.179 bits per weight, not the nominal 4.

From the file· summed from 1 file(s)From the file· KV per layer

Parameters

617M

Architecture

parakeet

null layers

Context

—

native (config.json)

License

cc-by-4.0

Shipped quantizations

● exact bytes, summed from published files

Quant	Size●	Exact bytes●	Effective bpw●	Tensors●	Publisher
Q4_K	0.37 GiB	397,689,376	5.159	—	cstr
Q4_K_M	0.44 GiB	476,390,816	6.179	—	handy-computer
Q5_K_M	0.50 GiB	539,714,976	7.001	—	handy-computer
Q6_K	0.56 GiB	600,902,048	7.795	—	handy-computer
Q8_0	0.68 GiB	729,687,456	9.465	—	handy-computer
F16	1.15 GiB	1,235,347,040	16.024	—	cstr
F16	1.15 GiB	1,235,969,568	16.032	—	handy-computer
F32	2.30 GiB	2,467,033,120	32.001	—	handy-computer

Measured

published by a third party, attributed below

Metric	Value	What it means
rtf	5406.7
RTFx	5406.7	Higher is better — audio seconds processed per second of compute.
Word error rate	1.25%	Lower is better — the share of words transcribed incorrectly.
Word error rate	2.88%	Lower is better — the share of words transcribed incorrectly.
Word error rate	2.64%	Lower is better — the share of words transcribed incorrectly.
Word error rate	5.78%	Lower is better — the share of words transcribed incorrectly.
Word error rate	8.55%	Lower is better — the share of words transcribed incorrectly.
Word error rate	16.47%	Lower is better — the share of words transcribed incorrectly.
Word error rate	13.58%	Lower is better — the share of words transcribed incorrectly.
Word error rate	14.34%	Lower is better — the share of words transcribed incorrectly.
Word error rate	8.45%	Lower is better — the share of words transcribed incorrectly.
Word error rate	3.41%	Lower is better — the share of words transcribed incorrectly.
Word error rate	6.65%	Lower is better — the share of words transcribed incorrectly.

Benchmarked· by open-asr-leaderboard-english-short-latest

RTFx measured by the Open ASR Leaderboard on a single datacenter GPU at a large batch size. It ranks models against each other; it says nothing about throughput on consumer hardware. We reproduce these figures with attribution; they are not ours and we have not verified the runs. Source: open-asr-leaderboard-english-short-latest.

Compare with

same modality, comparable size

vs nemotron-3.5-asr-streaming-0.6b vs parakeet-unified-en-0.6b vs parakeet-tdt-0.6b-v3 vs whisper-medium vs whisper-large-v3 vs canary-180m-flash All alternatives →Will it fit on my card? →

Why other calculators give a different number

A parameters × bits ÷ 8 estimate puts Q4_K_M at roughly 0.32 GiB. The real file is 0.44 GiB, because a quantization is a mixture and some tensors are always kept at higher precision.

Architecture

from config.json

Layers

—

Attention heads

—

KV heads

—

Head dim

—

Hidden size

—

Vocab

1,025

Sliding window

none

SWA period

—

MLA

Experts

—

Experts per token

—

use_sliding_window

—

Questions people ask

How much VRAM does parakeet-rnnt-0.6b need?: Q4_K_M is exactly 476,390,816 bytes (0.44 GiB) in weights. Add the KV cache, which depends on your context length, plus roughly half a gigabyte of runtime overhead.
Which quantization of parakeet-rnnt-0.6b should I use?: Q4_K_M is the usual default. Pick the largest quantization that fits your card at the context you actually need — the table above gives exact sizes for every one published.