Ollama basics: install, pull, serve and API

Introduction

Ollama packs model weights, runtime and an OpenAI-compatible API into one command. This lesson takes you from zero to a serving local model: install, pull, pick a size, customize with a Modelfile, call the API from code, and point OpenCode at it. Hub page: Ollama: local models the easy way.

Everything stays on your machine: no API bills, no data leaving, and it works offline.

Install on any OS

Linux and WSL2, one command:

curl -fsSL https://ollama.com/install.sh | sh

macOS and Windows: download the app from the official site:

https://ollama.com

Check it works:

ollama --version

Pull, run, manage models

Download once, reuse offline later:

ollama pull qwen3:8b

Chat in the terminal:

ollama run qwen3:8b

List downloaded models:

ollama list

Remove one you no longer need:

ollama rm qwen3:8b

Update weights later by pulling the same tag again. Inside ollama run, /bye exits the chat, /show prints the model configuration.

Which model and size

Rule of thumb: 7 to 8 billion parameters fits 8 GB of RAM and handles chat and drafting; 70 billion wants a serious GPU and pays off for code and reasoning. Start small — a fast small model you actually use beats a huge one you never start.

Browse the library for tags and sizes:

https://ollama.com/library

Judge with your own tasks, not leaderboards: three representative prompts, scored by you, separate good enough locally from needs the cloud in an afternoon.

Custom models with Modelfile

A Modelfile bakes a system prompt and parameters into a named model. File Modelfile beside your project:

FROM qwen3:8b PARAMETER temperature 0.2 SYSTEM You are a concise senior reviewer. Answer with code first.

Build and run it:

ollama create my-reviewer -f Modelfile

ollama run my-reviewer

Serve and HTTP API

ollama serve exposes two APIs on localhost port 11434: the native Ollama REST API and an OpenAI-compatible one, so coding assistants and scripts point at it by changing one base URL.

List models over HTTP:

curl http://localhost:11434/api/tags

One-shot generation, native API:

curl http://localhost:11434/api/generate -d "{\"model\": \"qwen3:8b\", \"prompt\": \"Why is the sky blue?\", \"stream\": false}"

Chat, OpenAI-compatible endpoint:

curl http://localhost:11434/v1/chat/completions -H "Content-Type: application/json" -d "{\"model\": \"qwen3:8b\", \"messages\": [{\"role\": \"user\", \"content\": \"Hi\"}]}"

Call it from code

Same code works against real OpenAI by changing base_url only:

from openai import OpenAI client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama") resp = client.chat.completions.create( model="qwen3:8b", messages=[{"role": "user", "content": "Explain LoRA in one paragraph."}], ) print(resp.choices[0].message.content)

Wire into OpenCode

With ollama serve running, OpenCode discovers local models automatically — open /models and pick ollama/qwen3:8b or your ollama/my-reviewer. No key needed. Pin it per project as in Connect a model, and use it as the private fallback when free cloud tiers disappear, as in Three models, one project. Keep one model loaded: swapping evicts RAM and stalls.

Env vars and keep-alive

Troubleshooting

Article author: Arthur Isaev