DeepSeek V4: Architecture, Benchmarks, and API Guide (2026)

TL;DR

DeepSeek V4 launched April 24, 2026 with two models. V4-Pro is a 1.6T-parameter MoE with 49B active per token. V4-Flash is a 284B MoE with 13B active. Both support 1M-token context and ship under the MIT license. V4-Pro-Max scores 80.6% on SWE-bench Verified and 93.5 on LiveCodeBench, the highest coding benchmark score of any model. V4-Flash costs $0.14/M input tokens. V4-Pro costs $1.74/M input tokens.

Key Specs at a Glance

V4-Pro

V4-Flash

Architecture: Hybrid Attention and Muon

V4 is not a simple scale-up of V3. The attention mechanism, optimizer, and training pipeline all changed. The result: V4-Pro requires only 27% of the single-token inference FLOPs and 10% of the KV cache compared to V3.2 in the 1M-token context setting.

1. Compressed Sparse Attention (CSA)

CSA compresses KV caches along the sequence dimension at a 4x compression rate, then applies sparse attention. A lightning indexer selects the top 1,024 most relevant compressed KV entries for each query. A sliding window of 128 tokens provides local context for each layer. This gives the model detailed, selective access to the most relevant parts of long contexts without the O(n²) cost of full attention.

2. Heavily Compressed Attention (HCA)

HCA applies a much more aggressive 128x compression rate but then performs dense attention over the compressed representation. This gives the model a cheap, global view of distant tokens in every layer. Where CSA is selective and detailed, HCA is broad and approximate. CSA and HCA layers are interleaved through the network, so the model alternates between focused retrieval and wide-angle context awareness.

3. Manifold-Constrained Hyper-Connections (mHC)

mHC upgrades residual connections for numerical stability in deep stacks. Standard residual connections can suffer from signal amplification or collapse at trillion-parameter scale. mHC constrains the mixing matrices to the Birkhoff Polytope using the Sinkhorn-Knopp algorithm, which preserves signal magnitude through the network.

4. Muon Optimizer

V4 switches from AdamW to the Muon optimizer for most parameters. DeepSeek reports faster convergence and more stable training at trillion-parameter scale. AdamW is retained for embeddings, the prediction head, and RMSNorm weights. Peak learning rate: 2.0e-4 for Pro, with cosine decay.

5. FP4 Quantization-Aware Training

FP4 quantization-aware training was applied to MoE expert weights and the indexer QK path during pre-training. This reduces memory requirements and enables more efficient inference without the quality loss that comes from post-training quantization.

Benchmark Performance

V4-Pro-Max is V4-Pro with extended reasoning tokens. It achieves the highest LiveCodeBench and Codeforces scores of any model and comes within 0.2 points of Claude Opus 4.6 on SWE-bench Verified.

V4-Pro-Max leads on coding benchmarks (LiveCodeBench, Codeforces) and achieves near-parity with Claude Opus 4.6 on SWE-bench. Claude holds a meaningful lead on HLE (40.0% vs 37.7%) and HMMT 2026 math (96.2% vs 95.2%). Gemini 3.1 Pro leads on MMLU-Pro and GPQA Diamond. The gap between these models is narrowing: the top four are all within single-digit percentage points on most benchmarks.

For Coding: How Good Is It?

V4-Pro-Max achieves the highest LiveCodeBench Pass@1 score of any model (93.5) and a Codeforces rating of 3206, ahead of GPT-5.4 xHigh (3168) and Gemini 3.1 Pro (3052). On SWE-bench Verified, it scores 80.6%, trailing Claude Opus 4.6 by 0.2 points.

Long-Context Code Tasks

The CSA+HCA attention mechanism is designed for long-context inference. At 1M tokens, V4-Pro uses 27% of the FLOPs and 10% of the KV cache compared to V3.2. CSA selects the 1,024 most relevant compressed KV entries per query, so the model focuses compute on the parts of the codebase that matter for each step. HCA maintains a compressed global view of the full context in every layer.

Cost Per Coding Task

V4-Pro-Max at $3.48/M output tokens delivers SWE-bench performance within 0.2 points of Claude Opus 4.6 at $75/M output tokens. That is a 21x cost reduction at near-identical coding benchmark performance. For teams running thousands of agentic coding tasks per day, this changes what is economically feasible.

V4-Flash at $0.28/M output is even cheaper. It approaches V4-Pro quality on general coding tasks with a 2-3 point gap, though it falls further behind on agentic coding (7-10 points on SWE-Pro and Terminal-Bench).

What Changed from DeepSeek V3

The efficiency gains are the most significant change. Despite being 2.4x larger overall and activating 1.3x more parameters per token, V4-Pro requires less compute and memory per inference step at long context lengths than V3.2. The CSA+HCA attention mechanism is the primary driver: by compressing and selecting KV entries rather than attending to all of them, V4-Pro scales sublinearly with context length.

API Pricing

V4-Pro-Max achieves SWE-bench parity with Claude Opus 4.6 at 1/21x the output token cost ($3.48 vs $75). V4-Flash is 268x cheaper than Claude on input tokens ($0.14 vs $15) and approaches V4-Pro quality on most tasks with a 2-3 point gap.

For a typical coding session (50K input, 10K output per request, 20 requests/day):

• V4-Flash: ~$0.20/day ($6/month)
• V4-Pro: ~$2.43/day ($73/month)
• Claude Opus 4.6: ~$30/day ($900/month)

V4-Pro vs V4-Flash: Which to Use

V4-Flash approaches V4-Pro quality on general tasks with a 2-3 point benchmark gap. The gap widens to 7-10 points on agentic coding tasks (SWE-Pro, Terminal-Bench) that require multi-step reasoning and tool use.

Limitations

Frequently Asked Questions

What models are in the DeepSeek V4 release?

Two models: V4-Pro (1.6T total, 49B active) and V4-Flash (284B total, 13B active). Both support 1M-token context and are MIT-licensed. V4-Pro-Max is V4-Pro with extended reasoning tokens for higher benchmark scores.

How does V4-Pro-Max compare to Claude Opus 4.6?

On SWE-bench Verified: V4-Pro-Max 80.6%, Claude Opus 4.6 80.8% (a 0.2 point gap). On LiveCodeBench: V4-Pro-Max 93.5, Claude Opus 4.6 Max 88.8. On HLE: Claude leads 40.0% vs 37.7%. On HMMT 2026 math: Claude leads 96.2% vs 95.2%. V4-Pro-Max costs $3.48/M output vs Claude's $75/M output.

What changed architecturally from V3?

Standard attention replaced with hybrid CSA+HCA. Muon optimizer replaced AdamW for most parameters. FP4 quantization-aware training applied to expert weights. mHC added for training stability. Training data increased from 14.8T to 33T tokens. The result: 27% of inference FLOPs and 10% of KV cache vs V3.2 at 1M context.

What does V4-Flash cost?

$0.14/M input tokens and $0.28/M output tokens. It approaches V4-Pro quality on general tasks (2-3 point gap) but falls further behind on agentic coding (7-10 point gap on SWE-Pro and Terminal-Bench).

Is V4 open source?

Yes. Both models are released under the MIT license with weights on Hugging Face. No commercial restrictions.

What is the context window?

1 million tokens for both V4-Pro and V4-Flash. The CSA+HCA hybrid attention mechanism makes this practical: V4-Pro uses only 10% of the KV cache that V3.2 would need at 1M context.