All models / Qwen3.5 122B-A10B
Qwen3.5 122B-A10B has 122B parameters (10B active per token, mixture-of-experts). At Q4_K_M with 8K context it needs about 75.3 GB of memory. The smallest graphics card that runs it fully on the GPU is about 96 GB, such as the RTX PRO 6000 Blackwell.
Check your own hardware| Quant | Bits per weight | Memory needed |
|---|---|---|
| Q8_0 | 8.5 | 130.9 GB |
| Q6_K | 6.56 | 101.3 GB |
| Q5_K_M | 5.69 | 88.1 GB |
| Q4_K_M | 4.85 | 75.3 GB |
| Q3_K_M | 3.91 | 60.9 GB |
| IQ2_M | 2.7 | 42.5 GB |
| Hardware | Memory | Best quant | Est. speed |
|---|---|---|---|
| RTX PRO 6000 Blackwell | 96 GB | Q5_K_M | 150+ tok/s |
| M4 Max 128GB | 128 GB | Q5_K_M | 46 tok/s |
| DGX Spark 128GB | 128 GB | Q5_K_M | 23 tok/s |
| Ryzen AI Max+ 395 128GB | 128 GB | Q5_K_M | 22 tok/s |
| M3 Ultra 256GB | 256 GB | Q8_0 | 46 tok/s |
| M3 Ultra 512GB | 512 GB | Q8_0 | 46 tok/s |