Free apps for running AI models on your own computer, sorted by what they do and which GPUs they support. Not sure your hardware can handle a model? Check your GPU first.
| Tool | Best for | GPUs | Runs on | Level |
|---|---|---|---|---|
| LM Studio | A polished desktop app for browsing, downloading and chatting with models | NVIDIA, AMD, Intel Arc, Apple Silicon, CPU only | Windows, macOS, Linux | Beginner |
| Jan | An open-source, offline ChatGPT-style desktop app | NVIDIA, AMD, Intel Arc, Apple Silicon, CPU only | Windows, macOS, Linux | Beginner |
| GPT4All | Private chat with your own documents on an ordinary PC | NVIDIA, AMD, Apple Silicon, CPU only | Windows, macOS, Linux | Beginner |
| AnythingLLM | Chatting with documents and building simple agents, all local | NVIDIA, AMD, Apple Silicon, CPU only | Windows, macOS, Linux | Beginner |
| KoboldCpp | Story writing and role-play with a single download | NVIDIA, AMD, Apple Silicon, CPU only | Windows, macOS, Linux | Intermediate |
| Tool | Best for | GPUs | Runs on | Level |
|---|---|---|---|---|
| Ollama | The simplest way to run models from the command line, with a local API other apps can use | NVIDIA, AMD, Apple Silicon, CPU only | Windows, macOS, Linux | Beginner |
| llama.cpp | Maximum control and the widest hardware support; the engine behind many other tools | NVIDIA, AMD, Intel Arc, Apple Silicon, CPU only | Windows, macOS, Linux | Advanced |
| vLLM | Fast serving for many users at once on Linux GPU servers | NVIDIA, AMD | Linux | Advanced |
| SGLang | High-throughput serving and structured outputs on Linux GPU servers | NVIDIA, AMD | Linux | Advanced |
| MLX LM | The fastest way to run models on Apple Silicon Macs | Apple Silicon | macOS | Intermediate |
| LocalAI | A drop-in local replacement for the OpenAI API, including images and audio | NVIDIA, AMD, Intel Arc, Apple Silicon, CPU only | Windows, macOS, Linux | Intermediate |
| Tool | Best for | GPUs | Runs on | Level |
|---|---|---|---|---|
| Text Generation WebUI | Trying many model formats and settings from one web interface | NVIDIA, AMD, Apple Silicon, CPU only | Windows, macOS, Linux | Intermediate |
| Open WebUI | A ChatGPT-like web interface for Ollama or any OpenAI-compatible server | Uses your engine's GPU support | Windows, macOS, Linux | Intermediate |
| Tool | Best for | GPUs | Runs on | Level |
|---|---|---|---|---|
| Continue | AI code completion and chat inside VS Code and JetBrains using local models | Uses your engine's GPU support | Windows, macOS, Linux | Intermediate |
| Cline | An autonomous coding agent in VS Code that can use local models | Uses your engine's GPU support | Windows, macOS, Linux | Intermediate |
| Aider | Pair programming in the terminal with git-aware edits | Uses your engine's GPU support | Windows, macOS, Linux | Intermediate |
| OpenCode | A terminal coding agent that can run on local models | Uses your engine's GPU support | Windows, macOS, Linux | Intermediate |
For most people, a desktop app like LM Studio or Jan, or Ollama if you are comfortable with one command in a terminal. All three are free and run on Windows, macOS and Linux.
LM Studio, Jan, llama.cpp, Ollama and LocalAI all support AMD GPUs. On Linux servers, vLLM and SGLang also support AMD.
MLX LM, Apple's framework for M-series chips, is usually fastest. LM Studio can use MLX too, and Ollama and llama.cpp use Apple's Metal GPU support.
Yes. Continue, Cline, Aider and OpenCode can all connect to a local engine like Ollama or LM Studio instead of a paid API. Agent-style coding works best with a capable model, roughly 24B parameters or more.