NVIDIA · consumer

GeForce RTX 3070 Ti

GeForce RTX 3070 Ti has 8 GB of VRAM at 608 GB/s — about 7.44 GiB usable after driver and compositor overhead. 608 of 2118 indexed models fit at 128K context with q8_0 KV.

Spec sheet· bandwidth, theoreticalFrom the file· fit from summed bytesPredicted· speed
Memory
8 GB
GDDR6X
Bandwidth
608 GB/s
256-bit bus
Tensor FP16
87 TF
dense
TDP
290 W
$599 MSRP
Context4K8K16K32K64K128K
KV cachef16q8_0q4_0quantizing the KV cache is a ~2× lever on the dominant term at long context
text 463vision language 71embedding 16video 8audio asr 31image 1audio tts 18

What fits at 128K context

largest quantization that fits, per model · 608 of 2118 indexed
ModelBest quantParamsWeights●KV●Total in memory◐Headroom◐tok/s◐
GLM-4.6V-FlashUD-IQ3_XXS10.3B3.94 GiB2.66 GiB7.44 GiB0.00 GiB64±12.9%
GLM-Z1-9B-0414UD-IQ3_XXS9.4B3.94 GiB2.66 GiB7.44 GiB0.00 GiB64±12.9%
GLM-4-9B-0414UD-IQ3_XXS9.4B3.94 GiB2.66 GiB7.44 GiB0.00 GiB64±12.9%
GLM-4.1V-9B-ThinkingUD-IQ3_XXS10.3B3.94 GiB2.66 GiB7.44 GiB0.00 GiB64±12.9%
SmolLM3-3BQ4_13.1B1.85 GiB4.78 GiB7.44 GiB0.00 GiB64±12.9%
OmniAtlas-Qwen3-30B-A3BI1-IQ1_M31.7B6.59 GiB0.00 GiB7.44 GiB0.00 GiB65±12.9%
Qwen3-Omni-30B-A3B-CaptionerI1-IQ1_M31.7B6.59 GiB0.00 GiB7.44 GiB0.00 GiB65±12.9%
Qwythos-9B-v2Q3_K_S9.7B4.48 GiB2.13 GiB7.44 GiB0.00 GiB64±12.9%
Tess-4-9BQ3_K_S9.7B4.48 GiB2.13 GiB7.44 GiB0.00 GiB64±12.9%
granite-4.1-3bUD-IQ3_XXS3.4B1.32 GiB5.31 GiB7.44 GiB0.00 GiB64±12.9%
glm4.1v-9b-base-sftI1-IQ3_XXS10.3B3.94 GiB2.66 GiB7.43 GiB0.01 GiB65±12.9%
glm-4v-9bQ5_K_M13.9B6.57 GiB0.00 GiB7.41 GiB0.03 GiB65±12.9%
Parable-Granite-4.1-3B-Claude-Fable-5I1-IQ3_XXS3.4B1.29 GiB5.31 GiB7.41 GiB0.03 GiB64±12.9%
granite-4.0-microIQ3_XXS3.4B1.29 GiB5.31 GiB7.41 GiB0.03 GiB64±12.9%
gte-largeQ6_K335M0.26 GiB6.38 GiB7.41 GiB0.03 GiB64±12.9%
granite-3.3-2b-instructIQ4_XS2.5B1.29 GiB5.31 GiB7.40 GiB0.04 GiB64±12.9%
granite-3.1-2b-instructIQ4_XS2.5B1.29 GiB5.31 GiB7.40 GiB0.04 GiB64±12.9%
granite-3.2-2b-instructIQ4_XS2.5B1.29 GiB5.31 GiB7.40 GiB0.04 GiB64±12.9%
granite-vision-3.2-2bIQ4_XS3.0B1.29 GiB5.31 GiB7.40 GiB0.04 GiB64±12.9%
granite-4.0-micro-baseQ2_K3.4B1.28 GiB5.31 GiB7.39 GiB0.05 GiB65±12.9%
GrammarCoder-7B-BaseI1-Q2_K7.6B2.82 GiB3.72 GiB7.39 GiB0.05 GiB65±12.9%
InternVL3_5-8BQ6_K_L8.5B6.54 GiB0.00 GiB7.39 GiB0.05 GiB65±12.9%
HunyuanVideo-1.5Q6_K8.3B6.54 GiB0.00 GiB7.39 GiB0.05 GiB65±12.9%
DeepHat-V1-7B-Heretic-AbliteratedI1-Q2_K7.6B2.81 GiB3.72 GiB7.38 GiB0.06 GiB65±12.9%
ShizhenGPT-7B-VLI1-Q2_K8.3B2.81 GiB3.72 GiB7.38 GiB0.06 GiB65±12.9%
DeepHat-V1-7BQ2_K7.6B2.81 GiB3.72 GiB7.38 GiB0.06 GiB65±12.9%
HuatuoGPT-o1-7BI1-Q2_K7.6B2.81 GiB3.72 GiB7.38 GiB0.06 GiB65±12.9%
MathSmith-DS-Qwen-7B-LongCoTI1-Q2_K7.6B2.81 GiB3.72 GiB7.38 GiB0.06 GiB65±12.9%
AstraGPTCoder-7BI1-Q2_K7.6B2.81 GiB3.72 GiB7.38 GiB0.06 GiB65±12.9%
Qwen2.5-Coder-7B-Instruct-Ghidra-v2I1-Q2_K7.6B2.81 GiB3.72 GiB7.38 GiB0.06 GiB65±12.9%
EsDrac-v1-7BI1-Q2_K7.6B2.81 GiB3.72 GiB7.38 GiB0.06 GiB65±12.9%
Hemlock-Apothecary-7B-GRPO-e3I1-Q2_K7.6B2.81 GiB3.72 GiB7.38 GiB0.06 GiB65±12.9%
openhands-lm-7b-v0.1I1-Q2_K7.6B2.81 GiB3.72 GiB7.38 GiB0.06 GiB65±12.9%
Hemlock2-Coder-7B-GRPOI1-Q2_K7.6B2.81 GiB3.72 GiB7.38 GiB0.06 GiB65±12.9%
shellwhiz-7bI1-Q2_K7.6B2.81 GiB3.72 GiB7.38 GiB0.06 GiB65±12.9%
Qwen2.5-Coder-7B-Instruct-abliteratedI1-Q2_K7.6B2.81 GiB3.72 GiB7.38 GiB0.06 GiB65±12.9%
Qwen2.5-Coder-7B-Instruct-OBLITERATED-advancedI1-Q2_K7.6B2.81 GiB3.72 GiB7.38 GiB0.06 GiB65±12.9%
Qwen-STEM-Specialist-7BI1-Q2_K7.6B2.81 GiB3.72 GiB7.38 GiB0.06 GiB65±12.9%
VulnLLM-R-7BI1-Q2_K7.6B2.81 GiB3.72 GiB7.38 GiB0.06 GiB65±12.9%
Garnet-OCR-7B-0422I1-Q2_K8.3B2.81 GiB3.72 GiB7.38 GiB0.06 GiB65±12.9%
UwU-7B-InstructI1-Q2_K7.6B2.81 GiB3.72 GiB7.38 GiB0.06 GiB65±12.9%
Video-R1-7BI1-Q2_K8.3B2.81 GiB3.72 GiB7.38 GiB0.06 GiB65±12.9%
HARC-Qwen2.5-7B-InstructI1-Q2_K7.6B2.81 GiB3.72 GiB7.38 GiB0.06 GiB65±12.9%
Qwen2.5-Coder-7B-AbliteratedI1-Q2_K7.6B2.81 GiB3.72 GiB7.38 GiB0.06 GiB65±12.9%
Bozdogan-7BI1-Q2_K7.6B2.81 GiB3.72 GiB7.38 GiB0.06 GiB65±12.9%
Qwen2.5-7B-Instruct-abliterated-v2Q2_K7.6B2.81 GiB3.72 GiB7.38 GiB0.06 GiB65±12.9%
Crazy-AI-ModelI1-Q2_K7.6B2.81 GiB3.72 GiB7.38 GiB0.06 GiB65±12.9%
turbo-ai-7bI1-Q2_K7.6B2.81 GiB3.72 GiB7.38 GiB0.06 GiB65±12.9%
DeepSeek-R1-Distill-Qwen-7B-abliterated-v2I1-Q2_K7.6B2.81 GiB3.72 GiB7.38 GiB0.06 GiB65±12.9%
Ghosty-7BI1-Q2_K7.6B2.81 GiB3.72 GiB7.38 GiB0.06 GiB65±12.9%
Qwen2.5-Coder-7B-InstructQ2_K7.6B2.81 GiB3.72 GiB7.38 GiB0.06 GiB65±12.9%
Bernini-MLLM-Qwen2.5-VL-7BQ2_K8.3B2.81 GiB3.72 GiB7.38 GiB0.06 GiB65±12.9%
Qwen2.5-Math-7B-InstructQ2_K7.6B2.81 GiB3.72 GiB7.38 GiB0.06 GiB65±12.9%
SP-7BI1-Q2_K7.6B2.81 GiB3.72 GiB7.38 GiB0.06 GiB65±12.9%
Qwen2.5-7B-InstructQ2_K7.6B2.81 GiB3.72 GiB7.38 GiB0.06 GiB65±12.9%
Qwen2.5-Coder-7B-Instruct-UncensoredI1-Q2_K7.6B2.81 GiB3.72 GiB7.38 GiB0.06 GiB65±12.9%
Qwen2.5-VL-7B-Instruct-abliteratedI1-Q2_K8.3B2.81 GiB3.72 GiB7.38 GiB0.06 GiB65±12.9%
DeepSeek-R1-Distill-Qwen-8B-AbliteratedI1-Q2_K7.6B2.81 GiB3.72 GiB7.38 GiB0.06 GiB65±12.9%
Qwen2.5-7BQ2_K7.6B2.81 GiB3.72 GiB7.38 GiB0.06 GiB65±12.9%
OREAL-DeepSeek-R1-Distill-Qwen-7BQ2_K7.6B2.81 GiB3.72 GiB7.38 GiB0.06 GiB65±12.9%
●From the file◐Predictedwhat these mean

Speed is modeled, not measured: decode is memory-bandwidth bound, so tokens per second is bytes read per token against achievable bandwidth. Mixture-of-experts models carry a wider band because only the routed experts are read each step, and few have been measured publicly.

Measured on this card

third-party benchmarks, aggregated
Workload◍MedianMiddle 50%Runs
Image generation10.81 it/s8.22–12.51504
Benchmarked· n=504

Aggregated from community-submitted runs, so the spread is wide by nature — it covers different models, resolutions, step counts and settings, not one controlled configuration. Read the middle 50% rather than the median alone. These figures are reproduced with attribution from vladmandic-sd-data-benchmark, which publishes no licence — so we display and link rather than redistribute them.

Questions people ask

What AI models can a GeForce RTX 3070 Ti run?
608 of 2118 indexed open-weight models fit a GeForce RTX 3070 Ti at 131,072 context with q8_0 KV cache, the largest being GLM-4.6V-Flash at UD-IQ3_XXS. That covers text, vision-language, image, video and speech models.
How much usable memory does a GeForce RTX 3070 Ti actually have?
Its nameplate is 8 GB, but about 7.44 GiB is available to a model once driver and compositor overhead is accounted for.
Is a GeForce RTX 3070 Ti fast for local AI?
Its memory bandwidth is 608 GB/s, and that figure — not teraflops — is what governs token generation speed. Capacity decides what you can run; bandwidth decides how fast it runs.