RunsLocalGPU checkerLocal AI toolsv1.9 · data Oct 15, 2026

All models / Nemotron 3.5 Lightning 30B-A3B

What GPU do I need to run Nemotron 3.5 Lightning 30B-A3B?

Nemotron 3.5 Lightning 30B-A3B has 30B parameters (3B active per token, mixture-of-experts). At Q4_K_M with 8K context it needs about 19.0 GB of memory. The smallest graphics card that runs it fully on the GPU is about 20 GB, such as the RX 7900 XT. Because only 3B parameters are active, it still runs usably with offload to system RAM on cards like the RTX 5080.

Check your own hardware

Memory by quantization (8K context)

Hardware that runs it fully in fast memory

Runs with offload to system RAM (slower)

RTX 5080 ~93 tok/s · RTX 5070 Ti ~92 tok/s · RTX 4080 Super ~87 tok/s · RTX 4080 ~86 tok/s · RTX 4070 Ti Super ~84 tok/s · RX 9070 XT ~83 tok/s · RX 9070 ~83 tok/s · RX 7800 XT ~82 tok/s · Arc A770 16GB ~80 tok/s · RX 6800 XT ~77 tok/s · RTX 5060 Ti 16GB ~73 tok/s · RX 9060 XT 16GB ~63 tok/s · RTX 4060 Ti 16GB ~60 tok/s · RTX 3080 Ti ~52 tok/s · RTX 3080 12GB ~52 tok/s · RTX 5070 ~50 tok/s · RTX 4070 Ti ~48 tok/s · RTX 4070 Super ~48 tok/s · RTX 4070 ~48 tok/s · Arc B580 ~47 tok/s · RX 7700 XT ~46 tok/s · RTX 2080 Ti ~45 tok/s · RTX 3060 12GB ~45 tok/s · RTX 3080 10GB ~42 tok/s

Assumes 32 GB of DDR5 system RAM for offload. Estimates only; real speeds usually land within about 20%.