All models / Llama 4 Scout
Llama 4 Scout has 109B parameters (17B active per token, mixture-of-experts). At Q4_K_M with 8K context it needs about 68.1 GB of memory. The smallest graphics card that runs it fully on the GPU is about 96 GB, such as the RTX PRO 6000 Blackwell.
Check your own hardware| Quant | Bits per weight | Memory needed |
|---|---|---|
| Q8_0 | 8.5 | 117.8 GB |
| Q6_K | 6.56 | 91.4 GB |
| Q5_K_M | 5.69 | 79.5 GB |
| Q4_K_M | 4.85 | 68.1 GB |
| Q3_K_M | 3.91 | 55.3 GB |
| IQ2_M | 2.7 | 38.8 GB |
| Hardware | Memory | Best quant | Est. speed |
|---|---|---|---|
| RTX PRO 6000 Blackwell | 96 GB | Q6_K | 77 tok/s |
| M3 Ultra 96GB | 96 GB | Q4_K_M | 48 tok/s |
| M4 Max 128GB | 128 GB | Q6_K | 24 tok/s |
| DGX Spark 128GB | 128 GB | Q6_K | 12 tok/s |
| Ryzen AI Max+ 395 128GB | 128 GB | Q6_K | 11 tok/s |
| M3 Ultra 256GB | 256 GB | Q8_0 | 27 tok/s |
| M3 Ultra 512GB | 512 GB | Q8_0 | 27 tok/s |