RunsLocalGPU checkerLocal AI toolsv1.8 · data Oct 14, 2026

All models / Llama 3.3 70B

What GPU do I need to run Llama 3.3 70B?

Llama 3.3 70B has 70.6B parameters. At Q4_K_M with 8K context it needs about 45.9 GB of memory. The smallest graphics card that runs it fully on the GPU is about 96 GB, such as the RTX PRO 6000 Blackwell.

Check your own hardware

Memory by quantization (8K context)

QuantBits per weightMemory needed
Q8_08.578.1 GB
Q6_K6.5661.0 GB
Q5_K_M5.6953.3 GB
Q4_K_M4.8545.9 GB
Q3_K_M3.9137.6 GB
IQ2_M2.726.9 GB

Hardware that runs it fully in fast memory

HardwareMemoryBest quantEst. speed
M4 Max 64GB64 GBQ4_K_M7.7 tok/s
Ryzen AI Max+ 395 64GB64 GBQ4_K_M3.6 tok/s
RTX PRO 6000 Blackwell96 GBQ8_014 tok/s
M3 Ultra 96GB96 GBQ6_K8.5 tok/s
M4 Max 128GB128 GBQ8_04.4 tok/s
DGX Spark 128GB128 GBQ8_02.2 tok/s
Ryzen AI Max+ 395 128GB128 GBQ8_02.0 tok/s
M3 Ultra 256GB256 GBQ8_06.6 tok/s
M3 Ultra 512GB512 GBQ8_06.6 tok/s

Runs with offload to system RAM (slower)

RTX 5090 ~2.9 tok/s · RTX 4090 ~1.9 tok/s · RTX 3090 Ti ~1.9 tok/s · RX 7900 XTX ~1.9 tok/s · RTX 3090 ~1.9 tok/s

Assumes 32 GB of DDR5 system RAM for offload. Estimates only; real speeds usually land within about 20%.