Screenshot from the article

Run ChatGPT-level AI locally using Ollama. No subscriptions, no data leaks, no internet required.

AI subscriptions are getting expensive.
Rate limits are annoying.

For the past few weeks, I’ve been running large language models locally on my laptop using Ollama — and honestly, for medium coding work, it’s good enough that I don’t miss Cursor or GPT most days.

This guide shows how to run LLMs locally with Ollama, step by step:

  • Installation
  • Best models for coding
  • Terminal usage
  • VS Code integration
  • Python scripting
  • Real examples developers actually care about

If you’ve ever searched for “run LLM locally”, “Ollama tutorial”, or “local AI coding assistant”, this is for you.

Why Run LLMs Locally Instead of Using ChatGPT or Claude?

Running local LLMs isn’t a gimmick anymore. It’s practical.

Benefits of Local AI Models

  • No subscription cost — one download, unlimited use
  • Privacy by default — code and prompts never leave your machine
  • Offline AI — works without internet
  • Predictable performance — no rate limits
  • Model choice — fast small models or deep reasoning models

Trade-offs (Be Honest)

  • Needs 8GB RAM minimum (16GB+ recommended)
  • Uses disk space (5–50GB per model)
  • Not ideal for ultra-long context tasks

If you’re a developer with a modern laptop, this is a very reasonable trade.

What Is Ollama?

Ollama is a local LLM runtime that:

  • Downloads models
  • Runs them locally
  • Exposes a CLI and API
  • Integrates with editors like VS Code

Think of it as Docker for LLMs, but simpler.

Official site: https://ollama.com

Step 1: Install Ollama (macOS, Linux, Windows)

macOS

brew install ollama

Or:

curl -fsSL https://ollama.com/install.sh | sh

Linux

curl -fsSL https://ollama.com/install.sh | sh

Windows

Download the installer from the website and run it.

Verify:

ollama --version

Start the service (optional):

ollama serve
Ollama usually auto-starts when you run a model.

Step 2: Best Ollama Models for Coding (2026)

Browse models here:
👉 https://ollama.com/library

Recommended Models

| Model              | Best For                   | RAM   |
| ------------------ | -------------------------- | ----- |
| `llama3.1:8b`      | General coding + reasoning | 16GB  |
| `qwen2.5-coder:7b` | Code generation            | 16GB  |
| `llama3.2:3b`      | Fast autocomplete          | 8GB   |
| `deepseek-r1:32b`  | Complex reasoning          | 32GB+ |

Download and Run a Model

ollama pull llama3.1:8b
ollama run llama3.1:8b

Example: Using Ollama Like ChatGPT (But Local)

>>> Explain Kubernetes in simple terms.

Output is immediate and private.

Exit with /bye or Ctrl+C.

List installed models:

ollama list

Step 3: Ollama CLI Examples (Very Useful)

One-shot Query

ollama run llama3.1:8b "Explain Docker multi-stage builds"

Pipe Input

cat app.py | ollama run qwen2.5-coder:7b "Review this code and suggest improvements"

DevOps Example

ollama run llama3.1:8b "Write a Terraform module for an S3 bucket with encryption"

This is where Ollama starts replacing cloud AI for daily work.

Step 4: VS Code Integration (Local Copilot Replacement)

Continue.dev + Ollama = local Claude/Copilot.

Setup

  1. Install Continue extension in VS Code
  2. Open config:
Cmd/Ctrl + Shift + P → Continue: Open config.json

Recommended Config

{
  "models": [
    {
      "title": "Llama 3.1 8B (Local)",
      "provider": "ollama",
      "model": "llama3.1:8b",
      "apiBase": "http://localhost:11434"
    }
  ]
}

What You Can Do

  • Highlight code → refactor
  • Ask for explanations
  • Generate tests
  • Convert scripts (Python ↔ Bash ↔ Go)
  • Debug errors

Example: Refactoring

Select a messy Bash script →
“Refactor and add comments”

Works surprisingly well.

Step 5: Using Ollama from Python (Automation-Friendly)

Install:

pip install ollama

Basic Python Example

import ollama
resp = ollama.chat(
    model="llama3.1:8b",
    messages=[{"role": "user", "content": "Explain zero-trust security"}]
)
print(resp["message"]["content"])

Streaming Example

for chunk in ollama.chat(
    model="llama3.1:8b",
    messages=[{"role": "user", "content": "Write a CI/CD pipeline"}],
    stream=True
):
    print(chunk["message"]["content"], end="")

Perfect for:

  • Internal tools
  • Chatbots
  • Code review bots
  • Offline assistants

Advanced Ollama Use Cases

Custom System Prompts

Create a Modelfile:

FROM llama3.1:8b
SYSTEM You are a senior DevOps engineer who prefers simple solutions.
ollama create devops-local -f Modelfile

Monitor Running Models

ollama ps

Health Check

curl http://localhost:11434

Hardware Requirements (Realistic)

  • 8GB RAM → 3B models
  • 16GB RAM → 7B–8B (sweet spot)
  • 32GB+ / GPU → 32B reasoning models

Ollama uses GPU automatically if available.

Final Thoughts: Is Running LLMs Locally Worth It?

For developers? Yes. Absolutely.

Local LLMs won’t fully replace cloud AI for everything, but for:

  • Coding
  • Scripting
  • DevOps
  • Learning
  • Privacy-sensitive work

Ollama is already good enough.

Start with:

  • llama3.1:8b
  • qwen2.5-coder:7b

Happy Coding!

📢 Have questions or feedback? Drop a comment below or connect with me on Twitter/X@spysood!

Originally published on Medium.