RunsLocalGPU checkerLocal AI toolsv1.8 · data Oct 14, 2026

All models / Mistral Small 3.1 24B

What GPU do I need to run Mistral Small 3.1 24B?

Mistral Small 3.1 24B has 24B parameters. At Q4_K_M with 8K context it needs about 16.4 GB of memory. The smallest graphics card that runs it fully on the GPU is about 20 GB, such as the RX 7900 XT.

Check your own hardware

Memory by quantization (8K context)

QuantBits per weightMemory needed
Q8_08.527.3 GB
Q6_K6.5621.5 GB
Q5_K_M5.6918.9 GB
Q4_K_M4.8516.4 GB
Q3_K_M3.9113.5 GB
IQ2_M2.79.9 GB

Hardware that runs it fully in fast memory

HardwareMemoryBest quantEst. speed
RX 7900 XT20 GBQ5_K_M28 tok/s
RTX 409024 GBQ6_K31 tok/s
RTX 3090 Ti24 GBQ6_K31 tok/s
RX 7900 XTX24 GBQ6_K29 tok/s
RTX 309024 GBQ6_K29 tok/s
M4 Pro 24GB24 GBQ4_K_M11 tok/s
RTX 509032 GBQ8_042 tok/s
M5 32GB32 GBQ6_K4.7 tok/s
M4 32GB32 GBQ6_K3.7 tok/s
M4 Max 36GB36 GBQ6_K13 tok/s
M4 Pro 48GB48 GBQ8_06.4 tok/s
M4 Max 64GB64 GBQ8_013 tok/s
Ryzen AI Max+ 395 64GB64 GBQ8_06.0 tok/s
RTX PRO 6000 Blackwell96 GBQ8_042 tok/s
M3 Ultra 96GB96 GBQ8_019 tok/s
M4 Max 128GB128 GBQ8_013 tok/s
DGX Spark 128GB128 GBQ8_06.4 tok/s
Ryzen AI Max+ 395 128GB128 GBQ8_06.0 tok/s
M3 Ultra 256GB256 GBQ8_019 tok/s
M3 Ultra 512GB512 GBQ8_019 tok/s

Runs with offload to system RAM (slower)

RTX 3080 Ti ~8.1 tok/s · RTX 3080 12GB ~8.1 tok/s · RTX 5070 ~7.7 tok/s · RTX 4070 Ti ~7.2 tok/s · RTX 4070 Super ~7.2 tok/s · RTX 4070 ~7.2 tok/s · Arc B580 ~7.0 tok/s · RX 7700 XT ~6.9 tok/s · RTX 2080 Ti ~6.6 tok/s · RTX 3060 12GB ~6.6 tok/s · RTX 3080 10GB ~6.0 tok/s · RTX 5060 Ti 8GB ~4.7 tok/s · RTX 5060 ~4.7 tok/s · RTX 3070 ~4.7 tok/s · RTX 3060 Ti ~4.7 tok/s · RTX 5050 ~4.5 tok/s · RTX 4060 Ti 8GB ~4.4 tok/s · RX 7600 ~4.4 tok/s · RTX 4060 ~4.3 tok/s

Assumes 32 GB of DDR5 system RAM for offload. Estimates only; real speeds usually land within about 20%.