RunsLocalGPU checkerLocal AI toolsv1.9 · data Oct 15, 2026

Home / Best models for 24 GB GPUs

Best local AI models for a 24 GB GPU

The best local AI models for a 24 GB GPU are the largest that fit fully in VRAM at Q4_K_M or better: Qwen3.6 35B-A3B (Q4_K_M, over 150 tokens/sec on an RTX 4090), Gemma 4 31B (Q5_K_M, about 28 tokens/sec on an RTX 4090), Nemotron 3.5 Lightning 30B-A3B (Q5_K_M, over 150 tokens/sec on an RTX 4090), Granite 4.2 30B (Q5_K_M, about 29 tokens/sec on an RTX 4090). Estimates assume 8K context.

Check your exact card and RAM

Runs fully on a 24 GB card

ModelBest quantMemory neededEst. speed (RTX 4090)
Qwen3.6 35B-A3B
MoE · coding		Q4_K_M22.2 GB150+ tok/s
Gemma 4 31B
general · vision		Q5_K_M23.0 GB28 tok/s
Nemotron 3.5 Lightning 30B-A3B
MoE · agentic		Q5_K_M22.1 GB150+ tok/s
Granite 4.2 30B
general		Q5_K_M21.6 GB29 tok/s
Qwen3.8 27B
coding		Q5_K_M20.9 GB31 tok/s
Qwen3.6 27B
coding		Q6_K23.2 GB27 tok/s
Gemma 4 26B A4B
MoE · vision		Q6_K21.8 GB150+ tok/s
Mistral Small 3.1 24B
general		Q6_K21.5 GB31 tok/s
gpt-oss-20b
MoE · reasoning		Q8_023.2 GB150+ tok/s
Phi-4 14B
reasoning		Q8_017.7 GB39 tok/s
Gemma 4 12B
general · audio		Q8_013.8 GB47 tok/s
Qwen3.5 9B
general		Q8_011.3 GB59 tok/s
Gemma 4 E4B
small · audio		Q8_09.3 GB126 tok/s
Gemma 4 E2B
small		Q8_06.1 GB150+ tok/s
Ministral 3 3B
small		Q8_04.8 GB150 tok/s

24 GB graphics cards

RTX 4090RTX 3090 TiRTX 3090RX 7900 XTX

Other VRAM sizes

8 GB12 GB16 GB32 GB

Speeds use the RTX 4090 (1008 GB/s). Cards with the same VRAM but different bandwidth will be faster or slower in proportion.