Local AI tools / For AMD GPUs
11 free local AI tools run models directly on AMD GPUs: Ollama, LM Studio, Jan, GPT4All, AnythingLLM, llama.cpp, vLLM, SGLang, KoboldCpp, Text Generation WebUI and LocalAI. The easiest to start with are Ollama, LM Studio, Jan, GPT4All and AnythingLLM. On Linux servers, vLLM and SGLang also support AMD. Chat interfaces and coding agents such as Open WebUI, Continue and Cline work on any GPU because they connect to one of these engines.
Check which models your GPU can run| Tool | Best for | GPUs | Runs on | Level |
|---|---|---|---|---|
| Ollama | The simplest way to run models from the command line, with a local API other apps can use | NVIDIA, AMD, Apple Silicon, CPU only | Windows, macOS, Linux | Beginner |
| LM Studio | A polished desktop app for browsing, downloading and chatting with models | NVIDIA, AMD, Intel Arc, Apple Silicon, CPU only | Windows, macOS, Linux | Beginner |
| Jan | An open-source, offline ChatGPT-style desktop app | NVIDIA, AMD, Intel Arc, Apple Silicon, CPU only | Windows, macOS, Linux | Beginner |
| GPT4All | Private chat with your own documents on an ordinary PC | NVIDIA, AMD, Apple Silicon, CPU only | Windows, macOS, Linux | Beginner |
| AnythingLLM | Chatting with documents and building simple agents, all local | NVIDIA, AMD, Apple Silicon, CPU only | Windows, macOS, Linux | Beginner |
| llama.cpp | Maximum control and the widest hardware support; the engine behind many other tools | NVIDIA, AMD, Intel Arc, Apple Silicon, CPU only | Windows, macOS, Linux | Advanced |
| vLLM | Fast serving for many users at once on Linux GPU servers | NVIDIA, AMD | Linux | Advanced |
| SGLang | High-throughput serving and structured outputs on Linux GPU servers | NVIDIA, AMD | Linux | Advanced |
| KoboldCpp | Story writing and role-play with a single download | NVIDIA, AMD, Apple Silicon, CPU only | Windows, macOS, Linux | Intermediate |
| Text Generation WebUI | Trying many model formats and settings from one web interface | NVIDIA, AMD, Apple Silicon, CPU only | Windows, macOS, Linux | Intermediate |
| LocalAI | A drop-in local replacement for the OpenAI API, including images and audio | NVIDIA, AMD, Intel Arc, Apple Silicon, CPU only | Windows, macOS, Linux | Intermediate |
| Tool | Best for | GPUs | Runs on | Level |
|---|---|---|---|---|
| Open WebUI | A ChatGPT-like web interface for Ollama or any OpenAI-compatible server | Uses your engine's GPU support | Windows, macOS, Linux | Intermediate |
| Continue | AI code completion and chat inside VS Code and JetBrains using local models | Uses your engine's GPU support | Windows, macOS, Linux | Intermediate |
| Cline | An autonomous coding agent in VS Code that can use local models | Uses your engine's GPU support | Windows, macOS, Linux | Intermediate |
| Aider | Pair programming in the terminal with git-aware edits | Uses your engine's GPU support | Windows, macOS, Linux | Intermediate |
| OpenCode | A terminal coding agent that can run on local models | Uses your engine's GPU support | Windows, macOS, Linux | Intermediate |
RX 9070 XTRX 9070RX 9060 XT 16GBRX 7900 XTXRX 7900 XTRX 7800 XTRX 7700 XTRX 7600RX 6800 XTRyzen AI Max+ 395 128GBRyzen AI Max+ 395 64GB