Best local AI models for 16GB VRAM
Ranked by what actually fits at 32K context, computed from real file bytes.
A 16GB card gives you about 14.88 GiB to work with after driver overhead. 39 indexed models fit at 32K context — the largest being Voxtral-Small-24B-2507 at 24.3B parameters in Q2_K_L.
From the file· fit from summed bytesFrom the file· KV per layer
Fits in 16GB at 32K context
largest quantization that fits, per model
| Model | Modality | Best quant | Params○ | Total◐ | Headroom◐ |
|---|---|---|---|---|---|
| nemotron-3.5-asr-streaming-0.6b | speech recognition | F32 | 638M | 3.22 GiB | 11.66 GiB |
| parakeet-tdt-0.6b-v3 | speech recognition | F32 | 627M | 3.18 GiB | 11.70 GiB |
| whisper-medium | speech recognition | F32 | 764M | 3.69 GiB | 11.19 GiB |
| Voxtral-Mini-4B-Realtime-2602 | speech recognition | F16 | 4.4B | 12.33 GiB | 2.55 GiB |
| whisper-large-v3 | speech recognition | F16 | 1.5B | 3.74 GiB | 11.14 GiB |
| whisper-large-v3-turbo | speech recognition | F16 | 809M | 2.36 GiB | 12.52 GiB |
| Qwen3-ASR-1.7B | speech recognition | F16 | 2.3B | 5.23 GiB | 9.65 GiB |
| Voxtral-Small-24B-2507 | speech recognition | Q2_K_L | 24.3B | 14.81 GiB | 0.07 GiB |
| Qwen3-ASR-0.6B | speech recognition | F16 | 938M | 2.32 GiB | 12.56 GiB |
| GigaAM-v3 | speech recognition | F32 | 223M | 1.67 GiB | 13.21 GiB |
| parakeet-ctc-0.6b | speech recognition | Q8_0 | 609M | 11.23 GiB | 3.65 GiB |
| granite-speech-4.1-2b-nar | speech recognition | F16 | 2.3B | 8.65 GiB | 6.23 GiB |
| whisper-small | speech recognition | F32 | 242M | 1.75 GiB | 13.13 GiB |
| granite-speech-4.1-2b | speech recognition | F16 | 2.3B | 8.48 GiB | 6.40 GiB |
| whisper-large | speech recognition | F32 | 1.5B | 6.60 GiB | 8.28 GiB |
| Voxtral-Mini-3B-2507 | speech recognition | F16 | 4.7B | 13.29 GiB | 1.59 GiB |
| canary-1b-flash | speech recognition | F32 | 811M | 4.16 GiB | 10.72 GiB |
| whisper-large-v2 | speech recognition | F32 | 1.5B | 6.60 GiB | 8.28 GiB |
| canary-qwen-2.5b | speech recognition | F16 | 2.6B | 6.15 GiB | 8.73 GiB |
| granite-speech-4.1-2b-plus | speech recognition | F16 | 2.1B | 8.50 GiB | 6.38 GiB |
| Breeze-ASR-25 | speech recognition | F16 | 1.5B | 3.74 GiB | 11.14 GiB |
| nemotron-speech-streaming-en-0.6b | speech recognition | F32 | 618M | 3.15 GiB | 11.73 GiB |
| granite-4.0-1b-speech | speech recognition | F16 | 2.3B | 7.60 GiB | 7.28 GiB |
| parakeet-ctc-1.1b | speech recognition | F32 | 1.1B | 4.80 GiB | 10.08 GiB |
| parakeet-rnnt-1.1b | speech recognition | F32 | 1.1B | 4.83 GiB | 10.05 GiB |
| whisper-base | speech recognition | F32 | 73M | 1.12 GiB | 13.76 GiB |
| moonshine-streaming-medium | speech recognition | F32 | 266M | 2.85 GiB | 12.03 GiB |
| whisper-medium.en | speech recognition | F32 | 764M | 3.69 GiB | 11.19 GiB |
| parakeet-rnnt-0.6b | speech recognition | F32 | 617M | 3.14 GiB | 11.74 GiB |
| whisper-small.en | speech recognition | F32 | 242M | 1.75 GiB | 13.13 GiB |
| moonshine-streaming-small | speech recognition | F32 | 140M | 1.91 GiB | 12.97 GiB |
| whisper-tiny | speech recognition | F32 | 38M | 0.99 GiB | 13.89 GiB |
| MOSS-Transcribe-Diarize | speech recognition | F32 | 909M | 7.66 GiB | 7.22 GiB |
| GLM-ASR-Nano-2512 | speech recognition | BF16 | 2.3B | 5.51 GiB | 9.37 GiB |
| ARK-ASR-3B | speech recognition | F16 | 4.1B | 8.93 GiB | 5.95 GiB |
| whisper-base.en | speech recognition | F32 | 73M | 1.12 GiB | 13.76 GiB |
| moonshine-streaming-tiny | speech recognition | F32 | 44M | 1.16 GiB | 13.72 GiB |
| moonshine-base | speech recognition | F32 | 62M | 0.99 GiB | 13.89 GiB |
| Qwen3-ForcedAligner-0.6B | speech recognition | F16 | 918M | 2.56 GiB | 12.32 GiB |
This page models a generic 16GB accelerator, so it answers what fits rather than how fast it runs. For tokens per second you need a specific card — pick one from hardware, where bandwidth is known.