What AI models can the RTX PRO 6000 Blackwell run?

With 96 GB of VRAM, the RTX PRO 6000 Blackwell runs 19 of 22 popular open models fully in fast memory at Q4_K_M or better (8K context, assuming 32 GB of DDR5 system RAM). The largest that fit are Qwen3.5 122B-A10B (Q5_K_M), gpt-oss-120b (Q5_K_M), Llama 4 Scout (Q6_K). Fastest: Gemma 4 E2B at over 150 tokens/sec and Qwen3.6 35B-A3B at over 150 tokens/sec. 1 more runs with offload to system RAM, but slower.

Check with your own RAM and context length

Model	Result	Best quant	Memory needed	Est. speed
Gemma 4 E2B small	Runs on GPU	Q8_0	6.1 GB	150+ tok/s
Qwen3.6 35B-A3B MoE · coding	Runs on GPU	Q8_0	38.2 GB	150+ tok/s
gpt-oss-120b MoE · reasoning	Runs on GPU	Q5_K_M	84.3 GB	150+ tok/s
gpt-oss-20b MoE · reasoning	Runs on GPU	Q8_0	23.2 GB	150+ tok/s
Ministral 3 3B small	Runs on GPU	Q8_0	4.8 GB	150+ tok/s
Gemma 4 26B A4B MoE · vision	Runs on GPU	Q8_0	27.9 GB	150+ tok/s
Gemma 4 E4B small · audio	Runs on GPU	Q8_0	9.3 GB	150+ tok/s
Qwen3.5 122B-A10B MoE	Runs on GPU	Q5_K_M	88.1 GB	150+ tok/s
Qwen3.5 9B general	Runs on GPU	Q8_0	11.3 GB	104 tok/s
Gemma 4 12B general · audio	Runs on GPU	Q8_0	13.8 GB	84 tok/s
Llama 4 Scout MoE · vision	Runs on GPU	Q6_K	91.4 GB	77 tok/s
Phi-4 14B reasoning	Runs on GPU	Q8_0	17.7 GB	69 tok/s
Mistral Small 3.1 24B general	Runs on GPU	Q8_0	27.3 GB	42 tok/s
Qwen3.6 27B coding	Runs on GPU	Q8_0	29.8 GB	37 tok/s
Qwen3.8 27B coding	Runs on GPU	Q8_0	30.6 GB	36 tok/s
Granite 4.2 30B general	Runs on GPU	Q8_0	31.9 GB	35 tok/s
Gemma 4 31B general · vision	Runs on GPU	Q8_0	33.8 GB	33 tok/s
Hermes 4.3 36B general	Runs on GPU	Q8_0	40.5 GB	28 tok/s
Llama 3.3 70B general	Runs on GPU	Q8_0	78.1 GB	14 tok/s
Qwen3.8-Flash-Next MoE	Needs offload	Q4_K_M	107.8 GB	77 tok/s
GLM-5.3-Flash MoE · coding	Too big	Q4_K_M	196.0 GB	—
DeepSeek V4.1-Flash MoE · coding	Too big	Q4_K_M	335.8 GB	—

Specs used: 96 GB of VRAM, 1792 GB/s memory bandwidth. Speeds are estimates for generating text with llama.cpp-style runtimes; real numbers usually land within about 20%.

Apps that support the RTX PRO 6000 Blackwell

Ollama LM Studio Jan GPT4All AnythingLLM llama.cpp vLLM SGLang KoboldCpp Text Generation WebUI LocalAI