Ollama turns your own computer into a private AI assistant. It downloads open-weight language models like Llama, Mistral, Gemma, and Qwen, runs them locally with GPU acceleration, and exposes them through a simple API on localhost:11434. Your prompts never leave your machine, there are no API fees, and it keeps working when your internet does not. This guide walks you from installation to your first conversation, plus how to pick the right model for your hardware and connect Ollama to your own apps.
Why run Ollama instead of cloud AI
Running models locally solves four problems that cloud services cannot. First, privacy: patient records, legal documents, company secrets, and personal notes never leave your machine. Second, cost: there are no per-token fees, which matters if you are building or testing at volume. Third, offline access: once a model is downloaded, Ollama works with no internet at all. Fourth, experimentation: you can pull and test dozens of open-weight models without paying for each call. The trade-off is simple — you are limited by your own RAM and GPU, and local models lag behind the very largest cloud models on the hardest reasoning tasks.
Check your system requirements
Ollama runs on macOS 14 (Sonoma) or later, Windows 10 or later, and any modern Linux distribution such as Ubuntu or Fedora. For hardware, 8 GB of RAM is the practical minimum and 16 GB or more is ideal. Small models (1B–3B parameters) run fine on CPU; models at 7B parameters and above are dramatically faster with a GPU — NVIDIA, AMD, and Apple Silicon are all supported. A rule of thumb from the community: a model fits comfortably in VRAM at roughly params × bits/8 × 1.2 GB of overhead, and Ollama uses Q4_K_M quantization (~4.5 effective bits per parameter) by default.
Install Ollama on your operating system
Installation is a single step on every platform.
macOS
Install with Homebrew:
brew install ollama
Or download the Ollama.dmg installer from ollama.com and drag it into Applications.
Windows
Download OllamaSetup.exe from the official site and accept the defaults — the installer sets everything up automatically and detects your GPU. PowerShell users can also run:
irm https://ollama.com/install.ps1 | iex
Linux
Run the official install script in your terminal:
curl -fsSL https://ollama.com/install.sh | sh
The installer enables Ollama as a background service automatically on most systems. Verify the install with:
ollama --version
curl http://localhost:11434
The second command should return "Ollama is running".
Run your first model in under five minutes
Ollama has one command that does everything: it downloads the model on first use, sets up the environment, and opens an interactive chat.
ollama run llama3.2:3b
The first run downloads a few gigabytes of model weights — this is the only step that needs the internet. After that, you can chat offline. Type /bye or press Ctrl-D to exit. To try another model later, just run it: ollama run mistral. Anything you have run before is cached locally, so subsequent launches are instant.
Pick the right model for your hardware
Downloading the biggest model available is the most common beginner mistake. Size up only as far as your memory allows — a model that does not fit will crash or crawl. These picks are solid starting points in 2026:
- 4 GB RAM or less: tinyllama, phi3:mini, llama3.2:1b
- 8 GB RAM: llama3.2:3b, gemma2:2b, qwen2.5-coder:1.5b
- 16 GB+ RAM with a GPU: mistral:7b, qwen2.5:7b, llama3.1:8b, gemma3:9b
In terms of download size: Llama 3.1 8B is about 4.7 GB, Mistral 7B about 4.1 GB, and Phi-3 Mini just 2.3 GB. For a quick start, llama3.2:3b (2 GB) is the best balance of quality and speed on ordinary laptops, while qwen2.5-coder variants are the go-to for a private coding assistant. DeepSeek R1 distilled versions (8B, 14B, 32B) are worth trying if you need step-by-step reasoning. Keep at least 10 GB of free disk space, since model weights accumulate fast.
Essential Ollama commands to memorize
You only need a handful of commands for daily use:
ollama list # show downloaded models
ollama pull llama3.1 # download a model without starting a chat
ollama run mistral # start an interactive session
ollama rm tinyllama # delete a model to free disk space
ollama serve # start the server manually (if not running as a service)
A note on tags: use the exact tag shown on the model library page — ollama pull deepseek-r1:32b downloads the 32B variant, while plain ollama pull deepseek-r1 pulls :latest, which may be a different size.
Use Ollama from your own apps via the API
This is where Ollama becomes genuinely useful for developers. It exposes an OpenAI-compatible API on port 11434, so you can point the OpenAI SDK at it with almost no code changes — you just change the base URL and pass any placeholder API key.
Set baseURL to http://localhost:11434/v1, keep the same chat completions call structure, and use the model name as Ollama sees it (for example, "llama3.1"). This one pattern unlocks Continue for coding assistance, Open WebUI for a ChatGPT-style browser interface, and custom scripts in any language. The REST endpoint at http://localhost:11434/api/generate also works directly with curl if you prefer plain HTTP.
For a nicer daily interface than the terminal, install Open WebUI — it gives you conversation history, model switching, and document uploads against your local models, all running in your browser while Ollama does the inference underneath.
Troubleshooting common problems
"Model not found" means you tried to run a tag you have not downloaded — fix it with ollama pull followed by the exact model name. If a GUI or container cannot connect, check Ollama is up with curl http://localhost:11434 and remember that Docker containers cannot reach host localhost — use your machine's LAN IP instead. Slow generation almost always means the model is larger than your hardware; drop from an 8B model to a 3B one and the speed difference is dramatic. And on Linux, enable the Ollama service at boot if you want it always available without a manual ollama serve.