RunsLocalGPU checkerLocal AI toolsv1.9 · data Oct 15, 2026

Home / Best models for 8 GB GPUs

Best local AI models for an 8 GB GPU

The best local AI models for an 8 GB GPU are the largest that fit fully in VRAM at Q4_K_M or better: Qwen3.5 9B (Q4_K_M, about 28 tokens/sec on an RTX 4060), Gemma 4 E4B (Q6_K, about 44 tokens/sec on an RTX 4060), Gemma 4 E2B (Q8_0, about 67 tokens/sec on an RTX 4060), Ministral 3 3B (Q8_0, about 40 tokens/sec on an RTX 4060). Mixture-of-experts models like Qwen3.6 35B-A3B (~31 tok/s) and Nemotron 3.5 Lightning 30B-A3B (~32 tok/s) also run usably with offload to 32 GB of system RAM. Estimates assume 8K context.

Check your exact card and RAM

Runs fully on a 8 GB card

ModelBest quantMemory neededEst. speed (RTX 4060)
Qwen3.5 9B
general		Q4_K_M6.9 GB28 tok/s
Gemma 4 E4B
small · audio		Q6_K7.4 GB44 tok/s
Gemma 4 E2B
small		Q8_06.1 GB67 tok/s
Ministral 3 3B
small		Q8_04.8 GB40 tok/s

Bigger models that run with offload

Qwen3.6 35B-A3B ~31 tok/s · Nemotron 3.5 Lightning 30B-A3B ~32 tok/s · Gemma 4 26B A4B ~27 tok/s · gpt-oss-20b ~32 tok/s

8 GB graphics cards

RTX 5060 Ti 8GBRTX 5060RTX 5050RTX 4060 Ti 8GBRTX 4060RTX 3070RTX 3060 TiRX 7600

Other VRAM sizes

12 GB16 GB24 GB32 GB

Speeds use the RTX 4060 (272 GB/s). Cards with the same VRAM but different bandwidth will be faster or slower in proportion.