How to Install Ollama on Linux, Windows, and macOS with Docker

Docker keeps the Ollama service easy to replace while its models stay in persistent storage. One model can use 2 GB to more than 70 GB. Downloading it again after rebuilding a container wastes an afternoon.

This setup uses Docker Engine on Linux and Docker Desktop on Windows or macOS. All three use a named volume and a port bound only to localhost.

What Is Ollama?

Ollama downloads and runs supported AI models on your own hardware. The official ollama/ollama image contains both the server and command-line client.

Local use has no per-token fee, but you still pay for hardware, power, and storage. Model downloads range from about 2 GB to more than 70 GB. Pick the model with more care than the container, which is fairly small.

Prerequisites

Make sure you have:

• A Linux host with Docker Engine 20.10 or later, or Docker Desktop on Windows or macOS
• Permission to run Docker commands
• At least 10 GB of free storage for a small model and the container image
• Enough system memory for the selected model
• Internet access for pulling the image and model files
• curl for command-line service testing, or a local web browser
• Docker Compose v2 only if you plan to use the optional Compose configuration

Requirement	Details
Linux test environment	Ubuntu 24.04 LTS with Docker Engine 27.x
Linux Docker minimum	Docker Engine 20.10
Windows	Docker Desktop using Linux containers
macOS	Docker Desktop
Ollama port	TCP 11434, bound to 127.0.0.1
Model storage	Named volume mounted at /root/.ollama
GPU	Optional; CPU operation works without GPU container support

A small model can use several gigabytes of memory while it runs. Leave room for other containers, virtual machines, and storage services. A larger SSD helps if you plan to keep several models. A full Docker data directory helps nobody, unless failed downloads are your hobby.

Step-by-Step Guide

Step 1: Confirm Docker Is Ready

Linux

These steps were tested on Ubuntu 24.04 LTS with Docker Engine 27.x. Check the Docker client and daemon:

docker --version
docker info

Expected output starts with a Docker version, then shows client and server details:

Docker version 27.x.x, build …
Client:
Version: 27.x.x
Server:
Containers: …

If docker info reports a socket permission error, the client can’t reach the Docker daemon. Use sudo docker for now, or follow Docker’s instructions to add your account to the docker group. That group grants root-level Docker access. Treat it that way.

Check Docker’s data path and the available space:

docker info --format '{{.DockerRootDir}}'
df -h /var/lib/docker

The second command assumes Docker uses /var/lib/docker. If the first command shows another path, give that path to df. Ten free gigabytes covers the image and one small model, but leaves little room for updates.

Windows

Install and open Docker Desktop. Wait until its dashboard says Docker Engine is running. Open PowerShell and run:

docker --version
docker info

Expected output has both client and server sections. Client details alone often mean Docker Desktop’s engine is stopped. Start Docker Desktop and try again.

macOS

Install Docker Desktop and open it from Applications. Wait for the engine to start. Open Terminal and run:

docker --version
docker info

Both commands should finish without a daemon connection error. Docker Desktop runs Linux containers in a virtual machine on macOS. Its memory and disk limits still apply when the Mac has spare resources.

Step 2: Pull the Official Ollama Image

Run the same command in Linux or macOS Terminal, or Windows PowerShell:

docker pull ollama/ollama

Expected output ends with a digest and success message:

Digest: sha256:…
Status: Downloaded newer image for ollama/ollama:latest

The latest tag is handy for an initial setup because Docker resolves it without another choice. It’s poor for controlled upgrades because the image can change. For repeatable deployments, choose a stable tag from the Ollama releases. Pin that tag during the upgrade step.

Step 3: Create Persistent Model Storage

Create a named Docker volume, then inspect it:

docker volume create ollama
docker volume inspect ollama

Expected output includes the volume name and Docker-managed mount point:

ollama
[
{
“Name”: “ollama”,
“Driver”: “local”,
“Mountpoint”: “…”
}
]

Ollama stores models and related data in /root/.ollama inside the container. Mount the named ollama volume there to keep the data when you replace the container. Containers are disposable. A 5 GB model download usually shouldn’t be.

Step 4: Start Ollama with Localhost-Only Access

Run this command on Linux or macOS:

docker run -d \
  --name ollama \
  --restart unless-stopped \
  -v ollama:/root/.ollama \
  -p 127.0.0.1:11434:11434 \
  ollama/ollama

In Windows PowerShell, use one line to avoid shell continuation differences:

docker run -d --name ollama --restart unless-stopped -v ollama:/root/.ollama -p 127.0.0.1:11434:11434 ollama/ollama

Each option has a job:

• -d runs the container in the background.
• --name ollama gives later commands a fixed container name.
• --restart unless-stopped restarts Ollama with Docker or the host, unless you stopped it yourself.
• -v ollama:/root/.ollama mounts the named volume at Ollama’s data directory.
• -p 127.0.0.1:11434:11434 publishes port 11434 only on the host’s loopback address.

Docker returns the new container ID:

7d48c9e2…

Confirm that Ollama is running and check its published address:

docker ps --filter name=ollama

Expected output has an Up status and this mapping:

127.0.0.1:11434->11434/tcp

That exact address matters. A mapping such as 0.0.0.0:11434->11434/tcp exposes the service through every host interface, subject to firewall rules.

On Windows or macOS, open Docker Desktop and select Containers > ollama. The container page shows its state, logs, and published port.

Step 5: Verify the Local Ollama Service

On Linux or macOS, query the service through its loopback address:

curl http://127.0.0.1:11434

On Windows PowerShell, call the executable by name. This avoids differences in how PowerShell versions handle the curl alias:

curl.exe http://127.0.0.1:11434

Expected response:

Ollama is running

You can also open http://127.0.0.1:11434 in a browser on the Docker host.

Other computers can’t reach this address. 127.0.0.1 accepts connections only from the Docker host. Keep that limit unless you’ve set up authentication, firewall rules, and encrypted access. Ollama’s API is useful, but an open model endpoint on the LAN is an avoidable surprise.

Step 6: Pull a Small Supported Model

Download the llama3.2 model:

docker exec -it ollama ollama pull llama3.2

docker exec starts Ollama’s CLI inside the running container. The -it flags attach an interactive terminal so the progress display works. Expected progress ends with:

success

The model goes into the named volume, not the container’s writable layer. Its exact size depends on the variant in the Ollama library. Watch the first download if disk space is tight. Models consume most of the storage here.

Step 7: Test Model Generation

Send a short prompt:

docker exec -it ollama ollama run llama3.2 "Reply with exactly: Ollama works"

A successful response should contain:

Ollama works

The wording can vary because model output isn’t deterministic, even with a request for exact text. Generation should finish without connection, memory, or model-not-found errors.

CPU generation may take several seconds or longer, based on the processor and model. Slow output alone doesn’t mean the setup has failed. An out-of-memory error does.

Step 8: Inspect Version, Logs, and Installed Models

Check the Ollama version inside the container:

docker exec ollama ollama --version

List downloaded models:

docker exec ollama ollama list

Expected inventory resembles:

NAME ID SIZE MODIFIED
llama3.2:latest … … …

Check the listed size before adding another model. Docker’s container size tells only part of the story because models live in a separate volume.

Review the latest 100 log lines:

docker logs --tail 100 ollama

Follow new messages while you reproduce a problem:

docker logs --follow ollama

Press Ctrl+C after you capture the error. This stops the log output, not the Ollama container.

Step 9: Upgrade Ollama Without Losing Models

Open the Ollama GitHub releases page and choose an exact stable release. Skip release candidates unless this host is for prerelease tests. A pinned tag gives you a known version and a clear rollback path.

Enter that checked tag when prompted:

read -r -p "Verified stable Ollama tag: " OLLAMA_TAG
docker manifest inspect "ollama/ollama:${OLLAMA_TAG}"
docker pull "ollama/ollama:${OLLAMA_TAG}"

In PowerShell:

$OllamaTag = Read-Host "Verified stable Ollama tag"
docker manifest inspect "ollama/ollama:$OllamaTag"
docker pull "ollama/ollama:$OllamaTag"

docker manifest inspect confirms that the registry can find the exact tag before you stop anything. If it returns a manifest error, fix the tag and leave the current container running.

Warning: The next commands stop and remove the ollama container. They preserve the separately named ollama volume. Do not add a volume-removal command.

On Linux or macOS:

docker stop ollama
docker rm ollama
docker run -d \
  --name ollama \
  --restart unless-stopped \
  -v ollama:/root/.ollama \
  -p 127.0.0.1:11434:11434 \
  "ollama/ollama:${OLLAMA_TAG}"

On Windows PowerShell:

docker stop ollama
docker rm ollama
docker run -d --name ollama --restart unless-stopped -v ollama:/root/.ollama -p 127.0.0.1:11434:11434 "ollama/ollama:$OllamaTag"

Check the new container and its model list:

docker exec ollama ollama --version
docker exec ollama ollama list

The models should remain because the new container mounts the same ollama volume. If the list is empty, check the volume mount before you download anything again.

Configuration

These settings cover common setups:

Setting	Recommended value	Why
Image	ollama/ollama:VERIFIED_STABLE_TAG	Makes upgrades repeatable
Model volume	ollama:/root/.ollama	Preserves model data across replacement
Port mapping	127.0.0.1:11434:11434	Restricts access to the host
Restart policy	unless-stopped	Restarts after host or Docker restarts
Loaded-model limit	OLLAMA_MAX_LOADED_MODELS=1	Useful on memory-constrained GPU systems

To limit Ollama to one loaded model, rebuild the container with this environment variable:

docker stop ollama
docker rm ollama
docker run -d \
  --name ollama \
  --restart unless-stopped \
  -e OLLAMA_MAX_LOADED_MODELS=1 \
  -v ollama:/root/.ollama \
  -p 127.0.0.1:11434:11434 \
  ollama/ollama

OLLAMA_MAX_LOADED_MODELS=1 cuts memory pressure when requests switch between models. The trade-off is more load time when Ollama swaps one model for another.

Optional NVIDIA GPU Setup on Linux

NVIDIA GPU access needs a supported host driver and the NVIDIA Container Toolkit. Install the toolkit from NVIDIA’s repository, then set up Docker’s runtime:

sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker

Restarting Docker can briefly affect every container on the host. Check other workloads first if the machine runs more than Ollama.

Rebuild Ollama with GPU access:

docker stop ollama
docker rm ollama
docker run -d \
  --name ollama \
  --restart unless-stopped \
  --gpus=all \
  -v ollama:/root/.ollama \
  -p 127.0.0.1:11434:11434 \
  ollama/ollama

The --gpus=all flag gives the container access to every GPU exposed by the NVIDIA runtime. That’s handy on a single-GPU workstation. On a shared host, grant only the devices Ollama needs.

The NVIDIA Container Toolkit guide covers setup for each repository and driver. GPU support differs on Docker Desktop for Windows and macOS. These Linux runtime commands don’t apply there.

Optional AMD GPU Setup on Linux

AMD’s Docker setup needs supported hardware, matching ROCm host drivers, and the ROCm image. The container also needs access to the GPU device nodes:

docker stop ollama
docker rm ollama
docker run -d \
  --name ollama \
  --restart unless-stopped \
  --device /dev/kfd \
  --device /dev/dri \
  -v ollama:/root/.ollama \
  -p 127.0.0.1:11434:11434 \
  ollama/ollama:rocm

/dev/kfd provides the compute interface. /dev/dri exposes the GPU rendering devices. Check ROCm support for your exact GPU first. AMD model names can look close on a product page while having very different support.

Tips and Troubleshooting

Ollama Does Not Answer on Port 11434

Cause: The container is stopped, still starting, or has the wrong port mapping.

Fix:

docker ps -a --filter name=ollama
docker logs --tail 100 ollama
docker start ollama
curl http://127.0.0.1:11434

docker ps -a shows stopped containers, while plain docker ps doesn’t. Read the logs before restarting the service again. A memory or device error will survive wishful thinking.

If the published port is wrong, stop and remove only the container. Create it again with -p 127.0.0.1:11434:11434.

Docker Reports That the Container Name Is Already in Use

Cause: A running or stopped container is already named ollama.

Fix:

docker ps -a --filter name=ollama
docker start ollama

If you mean to replace the current container:

docker rm ollama

The container must be stopped before docker rm works. Removing it leaves the named volume intact, as long as you don’t add a volume deletion command.

Downloaded Models Disappeared

Cause: Ollama started without -v ollama:/root/.ollama, or someone deleted the named volume.

Check whether the volume exists, then inspect its details:

docker volume ls
docker volume inspect ollama

Create the container again with the correct mount. If the volume was deleted, Docker can’t rebuild its model files. Back up Docker volumes before a major host move. Downloading the same 40 GB model twice gets old quickly.

A Model Download Fails for Lack of Space

Check the host’s free space and Docker’s disk use:

df -h
docker system df

df -h shows filesystem space in readable units. docker system df splits use by images, containers, volumes, and build cache.

Choose a smaller supported model or give Docker more storage. Check every target before running broad prune commands. An unused-looking volume may hold the only copy of a model or database.

Another Computer Cannot Connect

This is expected with 127.0.0.1:11434:11434. The loopback bind accepts connections only from software on the Docker host.

A 2.5GbE switch or Cat6 cable won’t change that bind policy. Keep the local address until you’ve added authentication, firewall rules, and encrypted transport. Raw LAN access saves a few setup minutes and can cost far more later.

Remove Ollama but Keep Its Models

Stop and remove only the container:

docker stop ollama
docker rm ollama

Confirm that the named volume remains:

docker volume ls

To restore Ollama later, run the original docker run command with -v ollama:/root/.ollama. Docker will attach the saved model data to the new container.

Warning: The following command permanently deletes the downloaded model data stored in the named volume. Run it only when you explicitly want to erase that data.

docker volume rm ollama

Don’t use docker compose down -v during routine upgrades. The -v flag removes Compose-managed named volumes. That turns a quick container refresh into another model download queue.

Wrapping Up

Step	Action	Applies To
1	Verify Docker and host resources	All platforms
2–4	Pull the image, create storage, and start Ollama	All platforms
5–8	Verify, test, and inspect the service	All platforms
9	Replace the container using a verified stable tag	All platforms
Optional	Configure NVIDIA or AMD access	Supported Linux hosts

Ollama now listens only on localhost, and its model files live outside the replaceable container. Upgrades replace the service layer while the large downloads stay in place.

Setup takes about 15 to 30 minutes, plus model download time. CPU inference works but may be slow. GPU support adds driver work. Keep the loopback bind until remote access has proper controls; an open AI API isn’t worth the shortcut.