What AI models can the RX 9060 XT 16GB run?

With 16 GB of VRAM, the RX 9060 XT 16GB runs 7 of 22 popular open models fully in fast memory at Q4_K_M or better (8K context, assuming 32 GB of DDR5 system RAM). The largest that fit are gpt-oss-20b (Q4_K_M), Phi-4 14B (Q6_K), Gemma 4 12B (Q8_0). Fastest: gpt-oss-20b at about 88 tokens/sec and Gemma 4 E2B at about 79 tokens/sec. 4 more run with offload to system RAM, but slower.

Check with your own RAM and context length

Model	Result	Best quant	Memory needed	Est. speed
gpt-oss-20b MoE · reasoning	Runs on GPU	Q4_K_M	13.6 GB	88 tok/s
Gemma 4 E2B small	Runs on GPU	Q8_0	6.1 GB	79 tok/s
Ministral 3 3B small	Runs on GPU	Q8_0	4.8 GB	48 tok/s
Gemma 4 E4B small · audio	Runs on GPU	Q8_0	9.3 GB	40 tok/s
Qwen3.5 9B general	Runs on GPU	Q8_0	11.3 GB	19 tok/s
Phi-4 14B reasoning	Runs on GPU	Q6_K	14.2 GB	16 tok/s
Gemma 4 12B general · audio	Runs on GPU	Q8_0	13.8 GB	15 tok/s
Gemma 4 26B A4B MoE · vision	Needs offload	Q4_K_M	16.4 GB	69 tok/s
Qwen3.6 35B-A3B MoE · coding	Needs offload	Q4_K_M	22.2 GB	50 tok/s
Mistral Small 3.1 24B general	Lower quality fit	Q3_K_M	13.5 GB	16 tok/s
Qwen3.6 27B coding	Lower quality fit	Q3_K_M	14.3 GB	15 tok/s
Qwen3.8 27B coding	Lower quality fit	Q3_K_M	14.7 GB	14 tok/s
Granite 4.2 30B general	Lower quality fit	Q3_K_M	15.1 GB	13 tok/s
Gemma 4 31B general · vision	Needs offload	Q4_K_M	19.8 GB	5.7 tok/s
Hermes 4.3 36B general	Needs offload	Q4_K_M	23.9 GB	3.8 tok/s
Llama 3.3 70B general	Too big	Q4_K_M	45.9 GB	—
Llama 4 Scout MoE · vision	Too big	Q4_K_M	68.1 GB	—
gpt-oss-120b MoE · reasoning	Too big	Q4_K_M	72.0 GB	—
Qwen3.5 122B-A10B MoE	Too big	Q4_K_M	75.3 GB	—
Qwen3.8-Flash-Next MoE	Too big	Q4_K_M	107.8 GB	—
GLM-5.3-Flash MoE · coding	Too big	Q4_K_M	196.0 GB	—
DeepSeek V4.1-Flash MoE · coding	Too big	Q4_K_M	335.8 GB	—

Specs used: 16 GB of VRAM, 320 GB/s memory bandwidth. Speeds are estimates for generating text with llama.cpp-style runtimes; real numbers usually land within about 20%.

Apps that support the RX 9060 XT 16GB

Ollama LM Studio Jan GPT4All AnythingLLM llama.cpp vLLM SGLang KoboldCpp Text Generation WebUI LocalAI