Install Ollama and Run Your First Local Model

Ollama is the easiest way to run an open-weight model on your own computer. You install one program, run one command to download a model, and you get both a terminal chat and an HTTP API. This post takes you from nothing to a working local model in about ten minutes.

Step 1: install

On macOS and Windows, download the installer from the Ollama website (ollama.com) and run it. On Linux, the official install script sets up the binary and a background service:

curl -fsSL https://ollama.com/install.sh | sh

As with any script piped into a shell, you can download it first and read it if you prefer. On macOS you can also install with Homebrew using brew install ollama. Confirm the install worked:

ollama --version

Step 2: make sure the server is running

Ollama works as a background server that listens on port 11434. The desktop installers start it for you. On Linux, the install script registers a service. If you ever need to start it by hand, run ollama serve in a terminal and leave it open. Test it:

curl http://localhost:11434

You should see the text Ollama is running. If the connection is refused, the server is not up yet.

Step 3: download a model

Pick a small model for your first run so the download is quick. This example uses the 3B Llama 3.2 model, which is around 2 GB:

ollama pull llama3.2

You will see progress bars for several layers. Ollama stores models in a local directory, so the next pull of the same model is instant.

Step 4: chat

ollama run llama3.2

This opens an interactive prompt. Type a question and press Enter. The first answer takes a moment because the model is loading into memory; later replies are quicker. Type /bye to leave. If you pass a prompt as an argument, Ollama answers once and exits:

ollama run llama3.2 "List three uses of a hash map."

If you run ollama run for a model you have not pulled, Ollama downloads it first, so pulling separately is optional.

Step 5: try the API

curl http://localhost:11434/api/chat -d '{
  "model": "llama3.2",
  "messages": [{"role": "user", "content": "Why is the sky blue? One sentence."}],
  "stream": false
}'

The reply is JSON, and the answer is under message.content. Without "stream": false Ollama sends many small JSON lines as the text is generated; post 10 explains that format.

Where models are stored

By default models live in a per-user directory such as ~/.ollama/models on macOS and Linux desktop installs. On a Linux system service the location can differ. You can move it by setting the OLLAMA_MODELS environment variable, which is useful if your home disk is small. Post 40 covers the main variables.

Troubleshooting first runs

• Connection refused: the server is not running. Start the app or run ollama serve.
• Very slow answers: the model may be too large for your memory and spilling to CPU or disk. Try a smaller one and check ollama ps.
• Disk full: remove unused models with ollama rm <name>.
• Port already in use: another Ollama process is running. Stop it before starting a new one.

How to choose the first model

The best first model is one that fits comfortably in memory and answers quickly, because a fast model teaches you more than a slow, impressive one. A model with one to three billion parameters runs acceptably even on laptops without a GPU. A model with seven or eight billion parameters is a much better conversation partner, but it needs around five gigabytes of free memory once loaded. If your first run feels sluggish, do not assume Ollama is broken. Look at the size of the model first, and try the next size down. You can always move up later, and each model you pull is independent of the others.

What the background server does

It helps to understand the architecture, because it explains most of the behaviour you will see. The ollama command you type is only a client. It talks over HTTP to the server process, which does the real work: it loads model files, keeps them in memory, runs the inference engine and answers requests. That is why the same server can serve your terminal, a web interface and your own scripts at once. It is also why a model can stay loaded after your chat ends, and why changing server settings needs a restart of the server rather than of your terminal.

What to do next

Learn the day-to-day commands in the next post, then move on to the API and your own code. You now have a private LLM that works without an internet connection.