RunsLocalGPU checkerLocal AI toolsv1.8 · data Oct 14, 2026

Local AI tools / vLLM

vLLM: what it is and what it runs on

vLLM is a free, open-source engine for running AI models on your own hardware. It runs on Linux. It supports NVIDIA GPUs and AMD GPUs. Best for: fast serving for many users at once on Linux GPU servers.

Visit vLLM
TypeEngines and servers
Open sourceYes
Operating systemsLinux
GPU supportNVIDIA, AMD
Skill levelAdvanced
NotesBuilt for throughput rather than one person chatting. Linux with NVIDIA or AMD GPUs.

Which models can you run with vLLM?

That depends on your graphics card's memory, not the app. Pick your hardware:

RTX 5090RTX 5080RTX 5070 TiRTX 5070RTX 5060 Ti 16GBRTX 5060 Ti 8GBRTX 5060RTX 5050RTX 4090RTX 4080 SuperRTX 4080RTX 4070 Ti SuperRTX 4070 TiRTX 4070 SuperRTX 4070RTX 4060 Ti 16GBRTX 4060 Ti 8GBRTX 4060RTX 3090 TiRTX 3090RTX 3080 TiRTX 3080 12GBRTX 3080 10GBRTX 3070RTX 3060 TiRTX 3060 12GBRTX 2080 TiRTX PRO 6000 BlackwellDGX Spark 128GBRX 9070 XTRX 9070RX 9060 XT 16GBRX 7900 XTXRX 7900 XTRX 7800 XTRX 7700 XTRX 7600RX 6800 XTRyzen AI Max+ 395 128GBRyzen AI Max+ 395 64GB

Similar tools

Ollamallama.cppSGLangMLX LMLocalAI