Best Ollama Models: 12 Models Ranked for Coding, RAG, and Agent Tasks

Quick-Pick: Best Ollama Model by Task

If you know what you need, start here. Each pick is the strongest model for that specific use case that actually runs on the hardware most developers have.

Master Comparison: 12 Ollama Models Ranked

VRAM measured with Q4_K_M quantization at default context length. Actual usage varies with context size. KV cache adds 1-2GB at default lengths, 14GB+ at 32K context for 70B models.

1. Qwen2.5-Coder 32B: The Local Coding Benchmark King

Qwen2.5-Coder 32B Instruct is the model that made "local coding AI" a serious statement instead of a compromise. It matches GPT-4o on HumanEval and leads every open-source model on EvalPlus, LiveCodeBench, and BigCodeBench. It runs on a single RTX 4090.

The 32B model scores 75.2 on MdEval (multi-language code repair), ranking first among all open-source models. It handles 40+ programming languages well, including less common ones like Haskell and Racket. Available in 6 sizes (0.5B, 1.5B, 3B, 7B, 14B, 32B), so you can match the model to your GPU.

Running It

ollama pull qwen2.5-coder:32b
ollama run qwen2.5-coder:32b "Refactor this function to use async/await"

Hardware Reality

At Q4_K_M quantization, the 32B model needs 22GB VRAM. An RTX 4090 (24GB) handles it with room for a 4-8K context window. An RTX 3090 (24GB) also works. For larger contexts, you will need to drop to Q3_K_M or use the 14B variant. On a MacBook Pro with 36GB+ unified memory, it runs comfortably at full Q4 precision.

The 7B variant fits on any 8GB GPU and still leads HumanEval among 7-8B class models. If you have a laptop with a GTX 1660 or RTX 3060, start with qwen2.5-coder:7b.

2. DeepSeek-R1 (Distilled): Chain-of-Thought Reasoning, Locally

DeepSeek-R1 is a "thinking" model. It reasons through problems step by step before producing an answer, similar to OpenAI's o1 series. The distilled versions bring this capability to local hardware at sizes from 1.5B to 70B.

The full R1 model is 671B parameters and needs ~400GB VRAM with 4-bit quantization. Not practical for local use. But DeepSeek open-sourced distilled checkpoints at 1.5B, 7B, 8B, 14B, 32B, and 70B, trained using 800,000 samples from the full model. The distills retain the reasoning behavior at a fraction of the compute.

Which Distill Size?

• 8B (6GB VRAM): Entry-level reasoning. Noticeably better than Llama 3.1 8B on math and logic puzzles. Viable on laptop GPUs.
• 14B (12GB VRAM): Sweet spot for mid-range GPUs. Good enough for debugging complex logic, tracing through state machines, explaining algorithms.
• 32B (20GB VRAM): The one to run if you have an RTX 4090. Scores 72.6% LiveCodeBench and systematically traces through logic before proposing fixes. Competitive with GPT-4o on reasoning tasks.
• 70B (43GB+ VRAM): Needs dual GPUs or a Mac with 64GB+ unified memory. Exceeds GPT-4o on several benchmarks.

ollama pull deepseek-r1:32b
ollama run deepseek-r1:32b "Why does this recursive function stack overflow on inputs > 10000?"

3. Qwen3-Coder 30B: The Agentic Coding Specialist

Qwen3-Coder is not just a code generation model. It was trained with reinforcement learning on SWE-Bench and designed for multi-step agentic workflows: reading files, running tests, editing code across repositories. The 30B variant (30B total, 3.3B active per token via MoE) runs on consumer hardware.

The MoE architecture means only 3.3B parameters activate per token out of 30B total. This gives you the quality of a much larger model with the inference speed and memory footprint of a small one. Context extends to 256K tokens natively and up to 1M with YaRN extrapolation, making it viable for repository-scale tasks.

Agent Tasks

Qwen3-Coder is the model to run when your workflow involves: reading a file, understanding a bug report, searching for related code, editing multiple files, and running tests. The RL training on SWE-Bench means it learned the loop of "read, reason, edit, verify" rather than just "generate code." Tool calling works reliably through Ollama's API.

ollama pull qwen3-coder:30b
# Use with any OpenAI-compatible agent framework
curl http://localhost:11434/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen3-coder:30b",
    "messages": [{"role": "user", "content": "Find and fix the null pointer in src/auth.ts"}],
    "tools": [{"type": "function", "function": {"name": "read_file", "parameters": {"type": "object", "properties": {"path": {"type": "string"}}}}}]
  }'

4. Llama 3.3 70B: The RAG Workhorse

Meta trained Llama 3.3 70B to match the 405B model's performance at a fraction of the compute. It succeeded. Instruction following at 92.1 beats every model in its class. Math improved by 9.2 points over Llama 3.1 70B. The 128K context window makes it the natural choice for RAG pipelines where you need to stuff retrieved documents into the prompt.

For RAG, what matters is: can the model synthesize an answer from injected context without hallucinating? Llama 3.3 70B does this well. The 128K context means you can inject 20-30 documents without truncation. Instruction following ensures the model answers the question you asked, not a tangentially related one.

Hardware

43GB VRAM at Q4_K_M. No single consumer GPU handles this. Options: dual RTX 3090s (48GB total), a single RTX A6000 (48GB), or a MacBook Pro with 64GB+ unified memory. With CPU offload and 64GB+ system RAM, you get 2-5 tokens/second, which is usable for batch processing but painful for interactive chat. The 8B variant (llama3.1:8b) fits on any modern GPU and serves as a capable lighter alternative.

5. Gemma 4: Native Function Calling Under Apache 2.0

Gemma 4 launched April 2, 2026, and immediately became the strongest open model for agent tasks in the sub-32B range. Four sizes: E2B (effective 2B), E4B, 26B MoE, and 31B dense. The 26B MoE activates 4B parameters per token, giving agent-grade quality at edge-friendly compute.

The differentiator: native function calling, structured JSON output, and system instruction support. Other models bolt these on via prompt engineering. Gemma 4 was trained for them. This means fewer dropped tool calls, fewer malformed JSON responses, and more reliable multi-step workflows.

Sizes and Use Cases

• E2B: On-device agents, mobile apps. The smallest model in this list that reliably calls tools.
• E4B: Edge deployments, Raspberry Pi-class hardware with GPU offload.
• 26B MoE: The sweet spot. 4B active parameters, native multimodal (vision + text), fits on 16GB VRAM. Use this for local agent pipelines.
• 31B Dense: Maximum quality when you have 20GB+ VRAM. #3 on Arena AI.

ollama pull gemma4:26b
# Supports native tool calling
ollama run gemma4:26b "Search for files matching *.ts in the src directory"

6. DeepSeek Coder V2 16B: Coding on a Budget GPU

DeepSeek Coder V2 Lite (16B) is the best coding model for developers with 12-16GB GPUs who want more than what 7B models offer. It uses a Mixture-of-Experts architecture, activating a subset of its 16B parameters per token for efficient inference.

On an RTX 3090 with 24GB, it runs at higher quantization levels (Q5_K_M, Q6_K) with room for 32K+ context. On a 16GB laptop GPU, Q4_K_M fits with default context. The 128K context window is large enough for most single-file and small multi-file tasks.

Compared to Qwen2.5-Coder 7B, the 16B gives noticeably better output on complex multi-step generation. Compared to Qwen2.5-Coder 32B, it is clearly weaker but runs on half the VRAM. The right model for the RTX 3060 Ti / RTX 4060 tier.

ollama pull deepseek-coder-v2:16b
ollama run deepseek-coder-v2:16b "Write a binary search tree with delete operation in Rust"

7. Phi-4 14B: Math and STEM on 10GB

Microsoft's Phi-4 is the model that violates the "bigger is better" assumption. At 14B parameters, it outperforms many 30-70B models on mathematical reasoning, structured logic, and STEM tasks. The tradeoff: a 16K context window, much smaller than the 128K offered by Llama or Gemma models.

On MATH, Phi-4 scores 80.4% versus Llama 3.3 8B at 68.0% and Qwen 2.5 14B at 75.6%. It delivers the best results per GB of VRAM for analytical tasks in 2026. The Phi-4-reasoning variant pushes this further, exceeding DeepSeek-R1 Distill Llama 70B (a model 5x its size) on reasoning benchmarks.

When to Use It

Phi-4 is the pick for: data analysis pipelines, scientific computation assistance, algorithm design, and any task where step-by-step logical reasoning matters more than broad knowledge or long-context synthesis. It is not the model for RAG (16K context is too small) or creative writing (not its training focus).

8. Gemma 3 12B: Multimodal Vision on One GPU

Gemma 3 is Google's previous-generation open model, still widely used because Gemma 4 only launched days ago. The 12B variant hits a sweet spot: multimodal (text + image), 128K context, and fits on a single 12GB GPU at Q4.

Gemma 3's QAT (quantization-aware trained) variants preserve near-BF16 quality at 3x less memory. The 12B QAT model runs on 8GB VRAM while maintaining quality that previously required 24GB. For vision tasks like OCR, chart understanding, and screenshot analysis, Gemma 3 12B is more accessible than Llama 3.2 Vision (which starts at 11B but needs more VRAM for the vision encoder).

The 4B model is worth considering for edge deployments: ~3GB VRAM at Q4, 128K context, multimodal. It loses meaningfully on complex reasoning compared to the 12B, but for structured output generation and simple visual tasks, it works.

9. Mistral Small 3.2: Multilingual Quality at 24B

Mistral Small 3.2 is Mistral's 24B dense model positioned for production chat and multilingual tasks. It delivers higher language quality than similarly sized models, particularly for European languages, but at a speed penalty: 18.5 tokens/sec on 16GB VRAM versus 70+ tokens/sec for smaller models.

The use case for Mistral Small 3.2 is specific: you need a local model that handles French, German, Spanish, or Italian text well, and you care more about output quality than speed. For English-only tasks, Qwen2.5-Coder 32B or Llama 3.3 70B are better choices. For multilingual customer support bots, documentation translation, or code comments in non-English languages, Mistral Small is the pick.

10. CodeLlama 34B: The Legacy Workhorse

CodeLlama 34B was state-of-the-art in 2023-2024. In 2026, it ranks 10th. That is not a knock on the model. It is a sign of how fast this space moves. Qwen2.5-Coder 32B at 92.7% HumanEval makes CodeLlama 34B at 53.7% look dated.

There are still reasons to use it. CodeLlama has the most mature fine-tuned ecosystem. If you have a domain-specific fine-tune trained on CodeLlama 34B, switching to Qwen2.5-Coder means retraining. The Python-specialized variant (CodeLlama-Python) is still competitive for pure Python generation tasks. And the 7B model (5-6GB VRAM) remains a decent option for constrained hardware where even Qwen2.5-Coder 7B feels slow.

11. Llama 3.1 8B: The Default Starting Point

111 million downloads on Ollama. If local LLMs had a "hello world," this is it. Llama 3.1 8B runs on 6GB VRAM, handles conversation, summarization, light coding, and general Q&A at a quality level that would have been impressive from a cloud API two years ago.

It is not the best at anything specific. It is good enough at almost everything. That makes it the right model for: learning Ollama, prototyping pipelines before scaling to a larger model, running on machines with 8GB GPUs, and any task where you need a quick answer and do not need specialist coding or reasoning capability.

The Llama 3.2 variants add vision capability (image understanding) at 11B. If you need multimodal on low hardware, consider Llama 3.2 Vision 11B as an alternative, though Gemma 3 12B generally performs better on visual tasks.

12. Nomic Embed Text: The RAG Embedding Standard

Every RAG pipeline needs an embedding model. Nomic Embed Text is the default choice in the Ollama ecosystem: 274MB, 8,192 tokens per chunk, 768-dimensional vectors, and it outperforms OpenAI's text-embedding-ada-002 and text-embedding-3-small on both short and long-context retrieval tasks.

It runs on any hardware. A CPU-only machine handles it fine. The v2 MoE variant adds multilingual support if you are indexing documents in multiple languages.

ollama pull nomic-embed-text
# Generate embeddings via API
curl http://localhost:11434/v1/embeddings \
  -H "Content-Type: application/json" \
  -d '{
    "model": "nomic-embed-text",
    "input": "How does the authentication middleware handle expired tokens?"
  }'

VRAM Cheat Sheet: What Runs on Your Hardware

The single biggest factor in local LLM performance is whether the model fits entirely in VRAM. A model that fits in GPU memory is 5-10x faster than one that spills to system RAM. This table maps hardware tiers to the best models you can run at full GPU speed.

Connecting Ollama to Morph Infrastructure

Whichever model you pick from this list, Morph's infrastructure works with it. Ollama exposes an OpenAI-compatible API at http://localhost:11434/v1. Any tool that accepts an OpenAI base URL can point to Ollama instead.

What This Means in Practice

• Fast Apply: Morph's apply model takes LLM-generated diffs and applies them to your codebase. It works with any model producing the edit. Use Qwen2.5-Coder 32B locally for code generation, then route through Fast Apply for reliable file edits. The apply model handles the last-mile formatting, which is where local models sometimes stumble.
• WarpGrep: The RL-trained search subagent runs in its own context window and finds relevant code before your main model starts working. Pairing WarpGrep with a local Ollama model means the local model only sees pre-filtered, relevant code rather than wasting its limited context on irrelevant files.
• Sandbox: Morph's sandboxed execution environment lets your Ollama model run generated code safely. The model writes code locally, Sandbox executes it in isolation, and the results flow back.

from openai import OpenAI

# Connect to local Ollama
client = OpenAI(
    base_url="http://localhost:11434/v1",
    api_key="ollama"  # Required but unused
)

# Use any Ollama model
response = client.chat.completions.create(
    model="qwen2.5-coder:32b",
    messages=[{"role": "user", "content": "Refactor auth.ts to use JWT instead of sessions"}]
)

# Same client works with Morph's API
morph_client = OpenAI(
    base_url="https://api.morphllm.com/v1",
    api_key="your-morph-key"
)

# Fast Apply takes the diff from any model
apply_response = morph_client.chat.completions.create(
    model="morph-v3-fast",
    messages=[{
        "role": "user",
        "content": f"<code>{original_code}</code>\n<update>{response.choices[0].message.content}</update>"
    }]
)

The combination of local inference (zero marginal cost, full privacy) with Morph's specialized infrastructure (reliable code application, intelligent search, safe execution) gives you the best of both worlds. Your code and prompts stay on your machine. Only the final diffs and search queries hit Morph's API.

Frequently Asked Questions

What is the best Ollama model for coding in 2026?

Qwen2.5-Coder 32B Instruct. It scores 92.7% on HumanEval (matching GPT-4o), runs on 24GB VRAM with Q4_K_M quantization, and leads open-source models on EvalPlus, LiveCodeBench, and BigCodeBench. For agentic coding, Qwen3-Coder 30B is the better choice because it was RL-trained on SWE-Bench and supports tool calling natively.

What is the best Ollama model for RAG?

Llama 3.3 70B for the generation model (128K context, strong instruction following, minimal hallucination on retrieved content). Nomic-embed-text for embeddings (274MB, 8,192 tokens/chunk). If 70B is too large for your hardware, Qwen2.5-Coder 32B handles code-heavy RAG well, and Llama 3.1 8B works as a lightweight alternative.

Which Ollama model supports tool calling for agents?

Qwen3 models have the most stable tool calling, rarely hallucinating calls or dropping parameters. Gemma 4 ships with native function calling trained into the model weights. Llama 3.3 also supports tool calling. Ollama's /api/chat endpoint handles tool definitions in the OpenAI format. Check Ollama's tool calling docs for the supported format.

How much VRAM do I need to run Ollama models?

At Q4_K_M quantization: 8B models need 6GB, 14B models need 10GB, 32B models need 20-22GB, 70B models need 43GB. The critical threshold is whether the model fits entirely in VRAM. Fully GPU-loaded models run 5-10x faster than those spilling to system RAM. For 8GB GPUs, Qwen2.5-Coder 7B or Phi-4-mini 3.8B are the best options.

Is Ollama's API compatible with OpenAI?

Yes. Ollama exposes OpenAI-compatible endpoints at http://localhost:11434/v1, including /chat/completions, /completions, /embeddings, and /models. Set base_url to http://localhost:11434/v1 and api_key to 'ollama' in the OpenAI Python client. This means Morph's Fast Apply, WarpGrep, and Sandbox work with any Ollama model without code changes.

What is the fastest Ollama model?

Speed depends on model size and whether it fits in VRAM. On 16GB VRAM, Phi-4-mini 3.8B and Qwen2.5-Coder 7B hit 40+ tokens/sec. Mistral Small 3.2 at 24B runs 18.5 tokens/sec on the same hardware. The general rule: a model that fits entirely in GPU memory generates 5-10x faster than one that spills to system RAM.

Can I use Ollama models with coding agents?

Yes. Any agent framework that supports OpenAI-compatible endpoints works with Ollama. Point the base URL to http://localhost:11434/v1. For agent-specific tasks, use Qwen3-Coder 30B (RL-trained for multi-step coding workflows) or Gemma 4 26B (native function calling). Morph's WarpGrep subagent pairs with any Ollama model to improve code search quality during agent runs.