Local AI tools / llama.cpp
llama.cpp is a free, open-source engine for running AI models on your own hardware. It runs on Windows, macOS and Linux. It supports NVIDIA GPUs, AMD GPUs, Intel Arc GPUs and Apple Silicon GPUs, and can also run on the CPU alone. Best for: maximum control and the widest hardware support; the engine behind many other tools.
Visit llama.cpp| Type | Engines and servers |
|---|---|
| Open source | Yes |
| Operating systems | Windows, macOS, Linux |
| GPU support | NVIDIA, AMD, Intel Arc, Apple Silicon, CPU only |
| Skill level | Advanced |
| Notes | Runs GGUF model files on almost any hardware, with GPU offload and a built-in server. |
RTX 5090RTX 5080RTX 5070 TiRTX 5070RTX 5060 Ti 16GBRTX 5060 Ti 8GBRTX 5060RTX 5050RTX 4090RTX 4080 SuperRTX 4080RTX 4070 Ti SuperRTX 4070 TiRTX 4070 SuperRTX 4070RTX 4060 Ti 16GBRTX 4060 Ti 8GBRTX 4060RTX 3090 TiRTX 3090RTX 3080 TiRTX 3080 12GBRTX 3080 10GBRTX 3070RTX 3060 TiRTX 3060 12GBRTX 2080 TiRTX PRO 6000 BlackwellDGX Spark 128GBRX 9070 XTRX 9070RX 9060 XT 16GBRX 7900 XTXRX 7900 XTRX 7800 XTRX 7700 XTRX 7600RX 6800 XTRyzen AI Max+ 395 128GBRyzen AI Max+ 395 64GBArc B580Arc A770 16GBM3 Ultra 512GBM3 Ultra 256GBM3 Ultra 96GBM4 Max 128GBM4 Max 64GBM4 Max 36GBM4 Pro 48GBM4 Pro 24GBM5 32GBM5 16GBM4 32GBM4 16GB