RunsLocalGPU checkerLocal AI toolsv1.8 · data Oct 14, 2026

All models / Hermes 4.3 36B

What GPU do I need to run Hermes 4.3 36B?

Hermes 4.3 36B has 36.2B parameters. At Q4_K_M with 8K context it needs about 23.9 GB of memory. The smallest graphics card that runs it fully on the GPU is about 32 GB, such as the RTX 5090.

Check your own hardware

Memory by quantization (8K context)

QuantBits per weightMemory needed
Q8_08.540.5 GB
Q6_K6.5631.7 GB
Q5_K_M5.6927.7 GB
Q4_K_M4.8523.9 GB
Q3_K_M3.9119.7 GB
IQ2_M2.714.2 GB

Hardware that runs it fully in fast memory

HardwareMemoryBest quantEst. speed
RTX 509032 GBQ5_K_M42 tok/s
M4 Max 36GB36 GBQ4_K_M11 tok/s
M4 Pro 48GB48 GBQ6_K5.5 tok/s
M4 Max 64GB64 GBQ8_08.5 tok/s
Ryzen AI Max+ 395 64GB64 GBQ8_04.0 tok/s
RTX PRO 6000 Blackwell96 GBQ8_028 tok/s
M3 Ultra 96GB96 GBQ8_013 tok/s
M4 Max 128GB128 GBQ8_08.5 tok/s
DGX Spark 128GB128 GBQ8_04.3 tok/s
Ryzen AI Max+ 395 128GB128 GBQ8_04.0 tok/s
M3 Ultra 256GB256 GBQ8_013 tok/s
M3 Ultra 512GB512 GBQ8_013 tok/s

Runs with offload to system RAM (slower)

RX 7900 XT ~7.3 tok/s · RTX 5080 ~4.7 tok/s · RTX 5070 Ti ~4.7 tok/s · RTX 4080 Super ~4.6 tok/s · RTX 4080 ~4.6 tok/s · RTX 4070 Ti Super ~4.5 tok/s · RX 9070 XT ~4.5 tok/s · RX 9070 ~4.5 tok/s · RX 7800 XT ~4.5 tok/s · Arc A770 16GB ~4.4 tok/s · RX 6800 XT ~4.3 tok/s · RTX 5060 Ti 16GB ~4.2 tok/s · RX 9060 XT 16GB ~3.8 tok/s · RTX 4060 Ti 16GB ~3.7 tok/s · RTX 3080 Ti ~3.4 tok/s · RTX 3080 12GB ~3.4 tok/s · RTX 5070 ~3.3 tok/s · RTX 4070 Ti ~3.2 tok/s · RTX 4070 Super ~3.2 tok/s · RTX 4070 ~3.2 tok/s · Arc B580 ~3.2 tok/s · RX 7700 XT ~3.2 tok/s · RTX 2080 Ti ~3.1 tok/s · RTX 3060 12GB ~3.1 tok/s

Assumes 32 GB of DDR5 system RAM for offload. Estimates only; real speeds usually land within about 20%.