RunsLocalGPU checkerLocal AI toolsv1.9 · data Oct 15, 2026

Home / Best models for 12 GB GPUs

Best local AI models for a 12 GB GPU

The best local AI models for a 12 GB GPU are the largest that fit fully in VRAM at Q4_K_M or better: Phi-4 14B (Q4_K_M, about 34 tokens/sec on an RTX 4070), Gemma 4 12B (Q6_K, about 31 tokens/sec on an RTX 4070), Qwen3.5 9B (Q8_0, about 29 tokens/sec on an RTX 4070), Gemma 4 E4B (Q8_0, about 63 tokens/sec on an RTX 4070). Mixture-of-experts models like Qwen3.6 35B-A3B (~41 tok/s) and Nemotron 3.5 Lightning 30B-A3B (~48 tok/s) also run usably with offload to 32 GB of system RAM. Estimates assume 8K context.

Check your exact card and RAM

Runs fully on a 12 GB card

ModelBest quantMemory neededEst. speed (RTX 4070)
Phi-4 14B
reasoning		Q4_K_M11.0 GB34 tok/s
Gemma 4 12B
general · audio		Q6_K10.8 GB31 tok/s
Qwen3.5 9B
general		Q8_011.3 GB29 tok/s
Gemma 4 E4B
small · audio		Q8_09.3 GB63 tok/s
Gemma 4 E2B
small		Q8_06.1 GB124 tok/s
Ministral 3 3B
small		Q8_04.8 GB75 tok/s

Bigger models that run with offload

Qwen3.6 35B-A3B ~41 tok/s · Nemotron 3.5 Lightning 30B-A3B ~48 tok/s · Gemma 4 26B A4B ~46 tok/s · gpt-oss-20b ~69 tok/s

12 GB graphics cards

RTX 5070RTX 4070 TiRTX 4070 SuperRTX 4070RTX 3080 TiRTX 3080 12GBRTX 3060 12GBRX 7700 XTArc B580

Other VRAM sizes

8 GB16 GB24 GB32 GB

Speeds use the RTX 4070 (504 GB/s). Cards with the same VRAM but different bandwidth will be faster or slower in proportion.