Ollama + Claude Code: How to Run Claude Code with Local Models (and Why You'd Want To)

Why Run Claude Code with Local Models

Claude Code against Anthropic’s API is the default. It works well. But three scenarios push developers toward local inference:

The tradeoff is speed. Local models on consumer hardware are significantly slower than cloud inference. The rest of this guide quantifies exactly how much slower, so you can decide whether the tradeoffs work for your situation.

Prerequisites

• Ollama v0.14+ (required for Anthropic API compatibility). Check with ollama --version.
• Claude Code CLI installed via curl -fsSL https://claude.ai/install.sh | bash (macOS/Linux) or irm https://claude.ai/install.ps1 | iex (Windows).
• Hardware: 16GB+ RAM (Apple Silicon) or 16GB+ VRAM (GPU) for local models. Cloud models have no hardware floor.

Quick Start: ollama launch claude

The fastest path. One command, no environment variables, no config files. Ollama handles everything.

# Launch Claude Code with Ollama's default coding model
ollama launch claude

# Or specify a model
ollama launch claude --model glm-4.7-flash

# Cloud model (no local hardware needed)
ollama launch claude --model qwen3.5:cloud

# Headless mode for scripts and CI
ollama launch claude --model glm-4.7-flash --yes -- -p "explain this codebase"

That’s it. ollama launch sets ANTHROPIC_AUTH_TOKEN, ANTHROPIC_BASE_URL, and ANTHROPIC_API_KEY automatically, then starts Claude Code pointed at your local Ollama instance.

Manual Configuration

If you prefer explicit control, or need to integrate with existing shell profiles, set the environment variables yourself.

# Set these in your terminal or add to ~/.zshrc / ~/.bashrc
export ANTHROPIC_AUTH_TOKEN=ollama
export ANTHROPIC_API_KEY=""
export ANTHROPIC_BASE_URL=http://localhost:11434

# Pull a model first
ollama pull glm-4.7-flash

# Start Claude Code with that model
claude --model glm-4.7-flash

Or inline, without touching your shell profile:

ANTHROPIC_AUTH_TOKEN=ollama \
ANTHROPIC_BASE_URL=http://localhost:11434 \
ANTHROPIC_API_KEY="" \
claude --model glm-4.7-flash

Settings.json Approach

You can also configure this in Claude Code’s settings file for persistence across sessions:

{
  "env": {
    "ANTHROPIC_AUTH_TOKEN": "ollama",
    "ANTHROPIC_API_KEY": "",
    "ANTHROPIC_BASE_URL": "http://localhost:11434"
  },
  "model": "glm-4.7-flash"
}

Which Models Work

Not every Ollama model is a good fit. Claude Code needs large context windows (64K+ tokens minimum) and tool calling support. Models without tool calling can generate text but cannot read files, run commands, or apply edits autonomously.

Local Models

Top pick: GLM-4.7-Flash. It uses mixture-of-experts architecture, so only 3B parameters activate per token despite being a 30B model. That means it runs on 16GB RAM or an RTX 3060. The 128K context window handles substantial codebases, and its tool calling scored 79.5% on agent benchmarks.

Cloud Models (via Ollama)

Ollama also proxies cloud models. These run on remote infrastructure but use the same Ollama interface. Free tier with generous rate limits.

Cloud Models: Free Inference Through Ollama

This is the setup most people actually want. You get Claude Code’s full agentic workflow, powered by models that rival Claude Sonnet, for $0.

# Qwen 3.5 — strong all-around coding model
ollama launch claude --model qwen3.5:cloud

# GLM-5 — competitive on code generation benchmarks
ollama launch claude --model glm-5:cloud

# Kimi K2.5 — good at multi-step reasoning
ollama launch claude --model kimi-k2.5:cloud

# 480B parameter Qwen3-Coder — largest available
ollama launch claude --model qwen3-coder:480b-cloud

Cloud models automatically get full context length. No need to configure num_ctx or worry about context truncation. Rate limits exist but are generous enough for normal development workflows.

Performance: Local vs Cloud

Real numbers from published benchmarks. The speed gap is the defining tradeoff of local inference.

Token Throughput

Real-World Task Comparison

From a published benchmark tracing entity relationships across a .NET monorepo:

The output quality was near-identical for that task. Both identified the same architectural patterns and produced equivalent analysis. But the cloud version finished in the time it takes to make coffee. The local version took longer than most meetings.

Coding Quality Benchmarks

From a controlled comparison running 50 coding tasks across 5 categories on an RTX 4070 Ti Super:

Local models score within 85-90% of Claude on single-file tasks like function generation and code explanation. The gap widens on multi-file reasoning, where Claude’s training and larger serving infrastructure give it a clear advantage. Bug detection also favors Claude: catching subtle issues requires the kind of deep reasoning that benefits from both model quality and inference speed.

Cost Breakeven

A heavy API user breaks even on GPU hardware in 6-8 months. Ollama cloud models are free but come with rate limits and send your code to external servers. For cost-sensitive developers without strict privacy requirements, cloud models through Ollama are the strongest value.

Hardware Requirements

If you want local inference without buying hardware, Ollama’s cloud models skip this entire section. Run ollama launch claude --model qwen3.5:cloud and your only requirement is a machine that can run Ollama itself.

Tool Calling and Limitations

Claude Code is not just a chat interface. It reads files, runs shell commands, applies edits, and searches code. All of that depends on tool calling: the model sends structured function calls, Claude Code executes them, and the results feed back into the conversation.

Without tool calling, Claude Code degrades to a plain text generator. You ask it to read a file and it describes what it would do instead of doing it.

What Works

• GLM-4.7-Flash has native tool calling support. 79.5% on agent benchmarks. Best local option.
• Qwen3-Coder supports tool calling. Slightly less reliable than GLM on complex multi-step chains.
• All Ollama cloud models (qwen3.5:cloud, glm-5:cloud, kimi-k2.5:cloud) have full tool calling.

What Doesn’t

• Most 7B models have weak or no tool calling. They generate text, but Claude Code cannot use them for autonomous operations.
• Older Ollama versions (pre-0.14) do not support the Anthropic API at all. Streaming tool calls specifically require 0.14.3-rc1+.
• Quantized models at Q2/Q3 lose too much instruction-following ability for reliable tool use.

The Edit Accuracy Problem

Even models with tool calling struggle with edit accuracy. Claude Code’s edit system was optimized for Claude’s output format. Alternative models approximate the format but miss details: wrong line numbers, bad whitespace, mismatched context. Published measurements show raw model diffs land at 70-80% accuracy for non-Claude models vs 98% for Claude.

That 20-30% failure rate compounds. Over a 50-edit session, you spend more time fixing broken patches than writing code.

Privacy and Compliance

This is the strongest argument for local models. When inference happens on your machine, the compliance picture simplifies dramatically:

• No cross-border data transfer. Code never leaves your jurisdiction. Relevant for GDPR, data residency laws, and government contracts.
• No third-party processing. Eliminates the need for Data Processing Agreements with model providers.
• Right to erasure. Your code is never encoded in external model weights. There is nothing to delete because nothing was stored.
• Air-gapped environments. Defense contractors and healthcare organizations often cannot send code to any external API. Local models are the only option.

A practical pattern for mixed environments: use local models for sensitive codebases (PII handling, medical records, proprietary algorithms) and cloud models for open-source or public-facing work. Tools like claude-launcher make switching between -l (local) and -a (API) instant.

Troubleshooting

Claude Code says “connection refused”

# Check if Ollama is serving
curl http://localhost:11434/api/version

# If nothing responds, start it
ollama serve

# Verify the base URL matches
echo $ANTHROPIC_BASE_URL
# Should print: http://localhost:11434

Model just talks instead of acting

You ask Claude Code to read a file. It responds with “I would read the file and then...” instead of actually reading it. This means tool calling is not working.

• Verify your Ollama version supports streaming tool calls (0.14.3-rc1+).
• Switch to a model with confirmed tool support: GLM-4.7-Flash or any cloud model.
• Check that ANTHROPIC_AUTH_TOKEN is set to ollama, not an actual API key.

Role model requests fail

Claude Code tries to use “haiku” for background tasks and fails because there’s no haiku model in Ollama.

# Point all role models to your local model
export ANTHROPIC_DEFAULT_HAIKU_MODEL=glm-4.7-flash
export ANTHROPIC_DEFAULT_SONNET_MODEL=glm-4.7-flash
export ANTHROPIC_DEFAULT_OPUS_MODEL=glm-4.7-flash

# Or use claude-launcher for automatic remapping
npm install -g claude-launcher
claude-launcher -l

Context window too small

If the model truncates responses or loses track of files, it may be running with a smaller context than configured.

# Create a Modelfile with explicit context
cat > Modelfile << 'EOF'
FROM glm-4.7-flash
PARAMETER num_ctx 65536
EOF

ollama create glm-4.7-flash-64k -f Modelfile
claude --model glm-4.7-flash-64k

Slow generation

If tokens come out painfully slow (under 5/sec), you are likely running a model that exceeds your hardware.

• Drop to a smaller quantization: Q4_K_M instead of Q6_K.
• Reduce context length. 32K is faster than 128K even if the model supports both.
• Switch to GLM-4.7-Flash if using a dense model. MoE architecture uses 3B active parameters from a 30B model.
• Consider cloud models. qwen3.5:cloud runs at 30-60 tok/s with zero hardware load.

Frequently Asked Questions

The Bottleneck Is Not the Model

Whether you run Claude Code against Anthropic’s API, a local Ollama model, or a free cloud model through Ollama, the same two problems dominate the agent loop: edit accuracy and search efficiency.

Local models make the edit problem worse. Claude’s 98% edit accuracy drops to 70-80% with alternative models. That 20-30% failure rate means retry loops, wasted tokens, and manual patch-fixing. Search is the other bottleneck: every irrelevant file loaded into context dilutes the signal the model reasons over.

These problems are solvable at a different layer. Morph’s Fast Apply model intercepts edit operations and merges them with a purpose-built model trained specifically for code application, pushing accuracy back to 98% regardless of which model generated the diff. It streams at 10,500+ tokens per second, so even large file rewrites complete in under a second. WarpGrep handles code search with 8 parallel tool calls per turn, filtering results before they hit your context window.

Both work with Claude Code as an MCP server. The model backend, whether Anthropic, Ollama local, or Ollama cloud, does not matter. The apply and search operations route through Morph independently.