All models / DeepSeek V4.1-Flash
DeepSeek V4.1-Flash has 552B parameters (16B active per token, mixture-of-experts). At Q4_K_M with 8K context it needs about 335.8 GB of memory. It needs a large unified-memory machine such as the M3 Ultra 512GB, or a multi-GPU setup.
Check your own hardware| Quant | Bits per weight | Memory needed |
|---|---|---|
| Q8_0 | 8.5 | 587.6 GB |
| Q6_K | 6.56 | 453.7 GB |
| Q5_K_M | 5.69 | 393.7 GB |
| Q4_K_M | 4.85 | 335.8 GB |
| Q3_K_M | 3.91 | 270.9 GB |
| IQ2_M | 2.7 | 187.4 GB |
| Hardware | Memory | Best quant | Est. speed |
|---|---|---|---|
| M3 Ultra 512GB | 512 GB | Q4_K_M | 51 tok/s |