How to Set Up Ollama for Self-Hosted AI Code Review in CI/CD Pipelines

If your legal team has ever flinched at the phrase “send our source code to a third-party API,” you already have the business case for this article. This tutorial walks through installing Ollama, picking a coding model sized to real GPU hardware, and wiring it into a CI/CD pipeline so every pull request gets an AI-generated review. No code leaves your network at any point.

By the end, you’ll have a working GitHub Actions job that fetches a PR diff, sends it to a local Ollama instance running on a GPU server you control, and posts a structured review comment back to the pull request using GitHub’s own API. We’ll also cover realistic GPU sizing for a 15-25 developer team, and where this setup falls short of commercial tools like GitHub Copilot’s review features or CodeRabbit. This is a trade you make on purpose.

What is Ollama?

Ollama is an open-source runtime for running large language models on your own hardware. It handles model downloads, quantization, and GPU scheduling, and exposes everything through a local REST API on port 11434. That API is close enough to the OpenAI chat completions format that most tooling built for hosted AI APIs can be pointed at it with minor changes.

For code review specifically, that local API is the whole point. A CI job can call http://your-gpu-host:11434/api/generate the same way it would call a cloud AI vendor, except the request never crosses your network boundary. Ollama itself is free and open-source (the GitHub repo has no paywalled features). ollama.com separately sells optional hosted cloud tiers, but this entire architecture works without touching them.

Before You Begin

Make sure you have:

• A Linux server (or VM) with an NVIDIA GPU to act as the shared inference host: this guide assumes Ubuntu 24.04 LTS
• NVIDIA drivers installed (nvidia-smi should return a valid output); driver 550.x or later recommended
• At least 16 GB of GPU VRAM for a useful coding model (24 GB+ recommended; see sizing table below)
• Root or sudo access on the GPU host
• A GitHub organization (or GitLab/Gitea equivalent) with a repository you can add Actions workflows to
• A private network or VPN connecting your CI runners to the GPU host; self-hosted GitHub Actions runners are the simplest default path for this tutorial, though GitHub also documents ways for GitHub-hosted runners to reach private networks (via private networking/overlay network features); this guide sticks with a self-hosted runner for simplicity
• Basic familiarity with GitHub Actions YAML and either Python or Bash

Requirement	Details
GPU host OS	Ubuntu 24.04 LTS (or equivalent recent Linux distro)
GPU	NVIDIA, 16 GB VRAM minimum, 24-48 GB recommended for a team
Ollama version	0.9.x or later (ollama --version)
CI runner	Self-hosted GitHub Actions runner on the same network as the GPU host
Git host token	Fine-grained PAT or GITHUB_TOKEN with pull-requests: write permission

This architecture also works with GitLab CI and Gitea Actions: swap the GitHub REST API calls for the equivalent GitLab/Gitea endpoints. The Ollama side of the pipeline doesn’t change.

Step-by-Step Guide

Step 1: Understand the Reference Architecture

Before installing anything, it helps to see the whole request path. Every later step maps to a piece of this diagram, so a couple minutes here saves confusion later:

• A developer opens or updates a pull request.
• A GitHub Actions workflow triggers on pull_request events and runs on a self-hosted runner that lives on the same private network as your GPU box.
• The workflow job fetches the PR diff using the GitHub REST API.
• The job sends that diff, wrapped in a review prompt, to the local Ollama API (POST /api/generate) running on the GPU host.
• Ollama runs inference with a coding model (e.g., Qwen2.5-Coder) and returns a structured review.
• The job posts that review back to the pull request as a comment, again via the GitHub REST API.

Nothing in steps 3-6 touches a third-party host. The only external calls are to your own Git host’s API, which already has your code, and to your own GPU server. That’s the entire zero-egress guarantee. It’s an architecture decision, not a special Ollama feature, so it’s worth drawing out before you write any YAML.

Step 2: Install Ollama on the GPU Host

Pick the subsection for your GPU host’s OS. For a team CI pipeline, Linux is almost always the right call. It’s the only platform here that runs headless as a systemd service without a logged-in user session, which is exactly what you want for a “shared inference server” that just sits there answering requests. macOS and Windows sections are included for teams prototyping on a workstation or running a Windows-based build fleet.

Linux (recommended for the CI/GPU host)

Run the official install script on the box that has your NVIDIA GPU:

curl -fsSL https://ollama.com/install.sh | sh

The script detects your GPU, installs the CUDA-compatible build, and registers a systemd service. Confirm it’s running:

systemctl status ollama

● ollama.service – Ollama Service
Loaded: loaded (/etc/systemd/system/ollama.service; enabled; vendor preset: enabled)
Active: active (running) since Tue 2026-09-01 09:12:04 UTC; 8s ago
Main PID: 41822 (ollama)

macOS (workstation / prototyping)

Install via Homebrew, or download the .app bundle from ollama.com/download:

brew install ollama

Launch the app once from /Applications; it registers itself as a menu-bar background service. On Apple Silicon, Ollama uses unified memory instead of discrete VRAM. That’s convenient for prototyping on your laptop, but not something you’d put behind a team’s CI pipeline at scale.

Windows (build fleet / Windows Server host)

Download OllamaSetup.exe from the download page and run it. The installer sets up Ollama as a background service. There’s no dedicated Windows Server GUI install path yet, so this works the same on Windows 10/11 and Windows Server 2022.

Verify the install from PowerShell:

ollama --version

ollama version is 0.9.4

Step 3: Pull a Coding Model Sized to Your GPU

This is where hardware and model choice actually meet, and it’s worth slowing down for. Ollama’s model library has several purpose-built code models; for CI review, stick to one of two families:

• Qwen2.5-Coder: currently the strongest open coding model at most size tiers, good instruction-following for structured review output.
• CodeLlama: older, more conservative, slightly weaker at multi-file reasoning but a solid fallback if you’re already running Llama-family infrastructure.

Pull a model sized to your GPU’s VRAM (see the sizing table in the Configuration section below):

ollama pull qwen2.5-coder:14b

pulling manifest
pulling 8934d96d3f08… 100% ▕████████████████▏ 9.0 GB
pulling 62fbfd9ed093… 100% ▕████████████████▏ 182 B
pulling c156170b718e… 100% ▕████████████████▏ 11 KB
verifying sha256 digest
writing manifest
success

Confirm it’s registered locally:

ollama list

NAME ID SIZE MODIFIED
qwen2.5-coder:14b 7f2c9e1a8b3d 9.0 GB 2 minutes ago

Step 4: Expose the Ollama API to Your Private Network

By default, Ollama only listens on 127.0.0.1:11434. That means nothing outside the GPU box can reach it, including your CI runner, which is a problem given the whole point of this exercise. On Linux, create a systemd override to change the bind address:

sudo systemctl edit ollama

Add these lines in the editor that opens:

[Service]
Environment="OLLAMA_HOST=0.0.0.0:11434"

Save, then restart the service:

sudo systemctl daemon-reload
sudo systemctl restart ollama

Warning: 0.0.0.0 binds to every network interface on the host, including any public-facing one. If this box has a public IP, restrict access with ufw or your firewall of choice, allow port 11434 only from your CI runner’s subnet, never from 0.0.0.0/0.

sudo ufw allow from 10.20.0.0/24 to any port 11434 proto tcp

Step 5: Test the API Endpoint

Before wiring anything into CI, confirm the API responds from another machine on the network, ideally the box that will run as your self-hosted GitHub Actions runner:

curl http://10.20.0.15:11434/api/generate -d '{"model":"qwen2.5-coder:14b","prompt":"Say hello in one word.","stream":false}'

{"model":"qwen2.5-coder:14b","created_at":"2026-09-01T09:20:11Z","response":"Hello","done":true}

If that comes back, your GPU host is ready to serve review requests. If it times out, skip to the troubleshooting section. Nine times out of ten it’s the firewall rule from Step 4.

Step 6: Write the Review Script

The review logic doesn’t need to live inside the YAML workflow. Keep it in a script checked into the repo so it’s testable and versioned. Save this as scripts/ai_review.py:

import json
import os
import urllib.request

OLLAMA_URL = os.environ["OLLAMA_URL"]  # e.g. http://10.20.0.15:11434/api/generate
MODEL = os.environ.get("OLLAMA_MODEL", "qwen2.5-coder:14b")
GITHUB_TOKEN = os.environ["GITHUB_TOKEN"]
REPO = os.environ["GITHUB_REPOSITORY"]
PR_NUMBER = os.environ["PR_NUMBER"]

REVIEW_PROMPT = """You are a senior code reviewer. Review the following diff.
Respond with a short bullet list covering: likely bugs, missing tests,
and style issues. Be specific and reference file names and line numbers
where possible. Keep the review under 300 words.

DIFF:
{diff}
"""

def get_diff():
    with open("pr.diff", "r", encoding="utf-8") as f:
        return f.read()

def call_ollama(diff_text):
    payload = json.dumps({
        "model": MODEL,
        "prompt": REVIEW_PROMPT.format(diff=diff_text[:12000]),  # keep under context window
        "stream": False,
    }).encode("utf-8")
    req = urllib.request.Request(OLLAMA_URL, data=payload, headers={"Content-Type": "application/json"})
    with urllib.request.urlopen(req, timeout=120) as resp:
        body = json.loads(resp.read())
    return body["response"]

def post_comment(review_text):
    url = f"https://api.github.com/repos/{REPO}/issues/{PR_NUMBER}/comments"
    payload = json.dumps({"body": f"**Automated review (self-hosted Ollama)**\n\n{review_text}"}).encode("utf-8")
    req = urllib.request.Request(url, data=payload, method="POST", headers={
        "Authorization": f"Bearer {GITHUB_TOKEN}",
        "Accept": "application/vnd.github+json",
        "Content-Type": "application/json",
    })
    urllib.request.urlopen(req, timeout=30)

if __name__ == "__main__":
    diff = get_diff()
    review = call_ollama(diff)
    post_comment(review)
    print("Posted review comment to PR #" + PR_NUMBER)

This script deliberately avoids third-party SDKs: urllib ships with Python, which keeps the CI runner’s dependency footprint small. Notice the diff_text[:12000] truncation. It’s a crude guard against blowing past the model’s context window on large PRs, and yes, crude is the right word; more on that in Troubleshooting.

Step 7: Store the Token Used to Post Comments

The script needs a token with permission to comment on pull requests. If your workflow runs on a self-hosted runner inside the same repo, the built-in GITHUB_TOKEN usually has enough scope by default. For cross-repo posting, generate a fine-grained personal access token with Pull requests: Read and write and store it as a repository secret.

Go to your repository’s Settings > Secrets and variables > Actions, and click New repository secret. Name it REVIEW_BOT_TOKEN.

Step 8: Wire It Into a GitHub Actions Workflow

Create .github/workflows/ai-code-review.yml:

name: AI Code Review (Self-Hosted Ollama)

on:
  pull_request:
    types: [opened, synchronize, reopened]

jobs:
  review:
    runs-on: [self-hosted, gpu-network]  # label your self-hosted runner accordingly
    steps:
      - name: Checkout PR
        uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Generate diff against base branch
        run: |
          git fetch origin ${{ github.event.pull_request.base.ref }}
          git diff origin/${{ github.event.pull_request.base.ref }}...HEAD > pr.diff

      - name: Run Ollama-backed review
        env:
          OLLAMA_URL: http://10.20.0.15:11434/api/generate
          OLLAMA_MODEL: qwen2.5-coder:14b
          GITHUB_TOKEN: ${{ secrets.REVIEW_BOT_TOKEN }}
          PR_NUMBER: ${{ github.event.pull_request.number }}
        run: python3 scripts/ai_review.py

The runs-on: [self-hosted, gpu-network] line is the important part. It pins this job to a runner you control that’s on the same private network as the Ollama host. A GitHub-hosted runner has no route to 10.20.0.15, so this job would just hang and time out on a public runner.

If you don’t already have a self-hosted runner on your GPU host’s network, set one up with actions-runner on a small VM in the same subnet. It’s a $0 line item if you’re already running homelab infrastructure, since it’s just another service on hardware you own.

Step 9: Verify the Full Pipeline

Open a test pull request with a trivial change, then watch the Actions tab for the workflow run.

If the job succeeds, the PR should show a new comment from your bot account within roughly 10-30 seconds for a 14B model on a mid-range GPU.

That comment confirms the full loop: diff out, inference on your own GPU, structured review back into GitHub, with a Cat6 Ethernet Cable and UPS Battery Backup for reliability. Nothing left your network to make it happen.

Configuration

Picking a Model and GPU Size

Model choice and GPU VRAM are the same decision wearing two hats. Undersize the GPU and you’re stuck with a model too weak to catch real bugs. Oversize it and you’re paying for headroom a 15-25 developer team will never touch.

Model	VRAM needed	Realistic concurrent reviewers	Review quality notes
qwen2.5-coder:7b	8-10 GB	5-8 devs	Fast, decent for style/lint-level catches, misses subtler logic bugs
qwen2.5-coder:14b	16-20 GB	15-25 devs	Sweet spot for a mid-size team, good bug detection, reasonable latency
qwen2.5-coder:32b	24-40 GB	25-40 devs (with request queuing)	Best accuracy, noticeably slower per-request, needs a 24 GB+ card
codellama:13b	14-16 GB	10-15 devs	Solid fallback, weaker at multi-file context than Qwen2.5-Coder

For a 15-25 developer team, qwen2.5-coder:14b on a single 24 GB GPU (an RTX 4090 or an RTX A5000-class card) is the realistic sweet spot. Not every developer opens a PR at the same moment, so one GPU comfortably absorbs the review queue for a team that size. The GPU is busy for seconds per review, not sustained hours.

Key Environment Variables

Variable	Purpose	Default
OLLAMA_HOST	Interface/port the API binds to; must be non-localhost for CI runners to reach it	127.0.0.1:11434
OLLAMA_MODELS	Filesystem path where model weights are stored; point this at a fast NVMe drive	OS-specific default
OLLAMA_NUM_PARALLEL	Concurrent requests the server processes; raise this if several PRs land at once	provider default (usually 1-4)
OLLAMA_KEEP_ALIVE	How long the model stays loaded in memory between requests	5 minutes

Set OLLAMA_KEEP_ALIVE=30m on a dedicated CI GPU host. It avoids reloading a 9 GB model from disk every time a new PR triggers a review, which is the single biggest latency win available without touching hardware.

Cost and Scale Breakeven vs. Paid Per-Seat Tools

Hosted AI code review tools typically run $15-40 per developer per month. For a 20-developer team, that’s $3,600-$9,600 a year, recurring forever, scaling linearly as the team grows. That number gets uncomfortable fast once you actually run it.

Approach	Upfront cost	Annual recurring cost (20 devs)	Notes
Hosted SaaS (per-seat)	$0	$3,600-$9,600	No hardware to manage, but code leaves your network
Self-hosted Ollama (RTX 4090, 24 GB)	~$1,800-2,200 (GPU) + ~$800 (host box)	~$150-300 (power, at ~350W under load)	One-time hardware, near-zero marginal cost per additional dev
Self-hosted Ollama (used workstation-class GPU)	~$800-1,200	~$100-200	Good for a smaller team or an initial pilot

At 15-25 developers, self-hosting typically pays for itself inside the first 4-8 months compared to a mid-tier hosted plan. Every developer added after that costs effectively nothing extra; the GPU doesn’t care if it’s serving 15 or 22 people, only how many requests land in the same minute. Below roughly 8-10 developers, the math is closer, and running your own GPU box may not beat just buying seats.

Tips and Troubleshooting

The CI runner can’t reach the Ollama API

Why it happens: Ollama only listens on localhost by default, and your Actions runner is a different machine.

Fix: Confirm OLLAMA_HOST=0.0.0.0:11434 is set via the systemd override from Step 4, restart the service, and check the firewall rule allows the runner’s subnet:

sudo ss -tlnp | grep 11434

LISTEN 0 4096 0.0.0.0:11434 0.0.0.0:*

If that line shows 127.0.0.1:11434 instead of 0.0.0.0:11434, the override didn’t take. Re-run systemctl daemon-reload and systemctl restart ollama.

The model runs out of VRAM or falls back to CPU

Why it happens: You pulled a model tag too large for the card, and Ollama silently offloads layers to system RAM, tanking performance.

Fix: Check GPU memory usage during a request with nvidia-smi. If it’s maxed out, pull a smaller tag (qwen2.5-coder:7b instead of :14b) or move to a GPU with more VRAM. There’s no shame in starting smaller: a 7B model that actually fits in VRAM beats a 14B model swapping to CPU every single time.

Large diffs exceed the model’s context window

Why it happens: A PR touching 40 files generates a diff far larger than most coding models’ context window (often 8K-32K tokens depending on the tag).

Fix: Chunk the diff by file and review each file separately, or skip generated/vendored files before sending the diff (e.g., exclude package-lock.json, *.min.js). The truncation in the sample script (diff_text[:12000]) is a blunt version of this fix. For production use, split by file and summarize each chunk instead of hard-truncating.

Reviews are slow when multiple PRs land at once

Why it happens: A single Ollama instance processes a limited number of concurrent requests, gated by OLLAMA_NUM_PARALLEL and available VRAM.

Fix: Raise OLLAMA_NUM_PARALLEL if you have VRAM headroom, or let CI jobs queue naturally; a review landing 60 seconds late rarely matters. For teams pushing past 25 developers, consider a second GPU behind a simple round-robin script rather than over-provisioning one card.

Self-hosted runner container has no GPU access

Why it happens: If your Actions runner itself runs in a container, the GPU isn’t passed through by default.

Fix: This only matters if Ollama runs inside the same container as the runner, which this architecture doesn’t require. Ollama should run directly on the GPU host, called over the network. If you do containerize Ollama, install the NVIDIA Container Toolkit and launch with GPU passthrough enabled.

Limitations vs. Hosted AI Code Review Tools

Self-hosting buys data control and predictable cost, not feature parity. Be honest with your team about the trade-offs before you sell them on this:

• No fine-tuned review models on your data. Hosted tools like CodeRabbit or Qodo train and tune specifically for code review; a general-purpose coding model gives solid but less specialized output.
• No IDE-integrated inline suggestions out of the box: this pipeline posts a single PR comment, not line-by-line inline annotations, unless you build that mapping yourself.
• You own the uptime. If the GPU host reboots or the model server hangs, reviews stop until someone notices; there’s no vendor SLA.
• False positives happen. Treat the AI comment as advisory input for the human reviewer, not a merge gate. Nothing here should block a PR on its own.
• Context window limits are real. Very large or cross-cutting PRs need chunking logic that hosted tools have already solved for you.

Wrapping Up

You now have a self-hosted pipeline that pulls a PR diff, runs it through a local Ollama-served coding model, and posts a structured review straight to GitHub, with nothing leaving your network. For a 15-25 developer team, a single 24 GB GPU running qwen2.5-coder:14b handles this comfortably and pays for itself against per-seat SaaS pricing well inside a year.

It’s a trade, not a free upgrade: you give up inline suggestions and vendor-tuned models in exchange for full data control and near-zero marginal cost per developer. For teams under NDA, in regulated industries, or just tired of sending proprietary code to another company’s API, that trade is an easy call.

Step	Action	Applies To
1	Install Ollama on a Linux GPU host	Linux (primary), macOS/Windows (workstation)
2	Pull qwen2.5-coder:14b sized to your VRAM	All platforms
3	Set OLLAMA_HOST=0.0.0.0:11434 and restrict via firewall	Linux GPU host
4	Write the diff-review-comment script	CI runner
5	Store the PAT as a repo secret	GitHub
6	Wire the workflow to a self-hosted runner on the GPU network	GitHub Actions
7	Verify the PR comment appears	GitHub

Resources

• Ollama official site
• Ollama download page
• Ollama model library
• Ollama GitHub repository
• Ollama API reference
• Ollama release notes
• GitHub Actions documentation
• Cat6 Ethernet Cable
• UPS Battery Backup