Anthropic measured a multi-agent system beating a single agent by 90.2%. The same write-up says multi-agent systems burn about 15x the tokens of a chat interaction. Those are not two findings. They are one. Orchestration buys accuracy with tokens, and most teams budget for the first number without reading the second.
What Is AI Agent Orchestration?
AI agent orchestration is the runtime coordination of multiple AI agents working on a single task: deciding which agent runs, in what order, on what context, and what happens when one of them fails. The orchestrator owns control flow and context routing. It does not do the work itself.
The distinction that trips people up is orchestration versus framework. An agent framework is what you write with: the SDK that hands you a tool-calling loop, a prompt structure, and a way to declare tools. Orchestration is what happens when the thing runs: which agent gets called, what it is allowed to see, how a failed step is retried, how results merge back. Framework bugs surface at write time. Orchestration bugs surface under load, on the trajectory you did not test.
Concretely: a coding agent asked to add rate limiting to an API. A single agent greps the repo, reads twelve files, finds the middleware layer, writes the change, runs the tests. An orchestrated system dispatches a search agent to locate the middleware and return two file spans, hands those spans to a coding agent that never saw the ten irrelevant files, and runs a test agent on the result. Same task. The second system keeps the expensive model's context clean, which is the entire mechanism behind the accuracy gain.
“We found that a multi-agent system with Claude Opus 4 as the lead agent and Claude Sonnet 4 subagents outperformed single-agent Claude Opus 4 by 90.2% on our internal research eval.”
Read the mechanism, not just the number. The subagents were Sonnet 4, a smaller model than the Opus 4 baseline they beat. The win did not come from more intelligence. It came from context isolation: each subagent explored in its own window and returned conclusions, so the lead agent's context never filled with rejected paths.
Orchestration Patterns
Most production systems are one of three shapes, or a composition of them. The pattern you pick determines your cost curve and your failure modes more than your framework choice does.
Planner-worker
One lead agent decomposes the task and delegates subtasks to workers, each running in an isolated context window and returning only its result. The planner never sees the workers' intermediate reasoning. This is the pattern behind Anthropic's 90.2% result and the one most coding agents use for search. It adapts to novel inputs because the decomposition happens at runtime, and it costs a planning round trip before any real work starts.
Pipeline
A fixed sequence of stages, each agent transforming the previous stage's output. Extract, then classify, then summarize. There is no dynamic routing, which makes pipelines cheap to run, trivial to trace, and easy to unit test stage by stage. They break on inputs the sequence was not designed for, because no component has the authority to change the plan.
Parallel fan-out
N independent agents dispatched simultaneously against the same problem, with their returns merged. Fan-out buys wall-clock time when subtasks are genuinely independent, and its token cost scales linearly with the fan width whether or not the extra agents contribute. Cognition's search work is a concrete instance: SWE-grep issues 8 parallel tool calls per turn across 4 turns, 3 exploration turns and 1 answer turn.
| Pattern | Control flow | Token cost | Fails when |
|---|---|---|---|
| Planner-worker | Decided at runtime by the lead agent | Planning round trip plus workers | The planner misjudges the decomposition |
| Pipeline | Fixed at design time | Lowest, one pass per stage | Input does not fit the fixed sequence |
| Parallel fan-out | All branches dispatched at once | Linear in fan width | Subtasks were not actually independent |
AI Agent Orchestration Platforms and Tools
These are the libraries and SDKs people actually run orchestration on. Each entry says what coordination it gives you at runtime, which is the only axis that matters here. For a full feature comparison of these as development libraries, that belongs on the agent framework comparison.
LangGraph
Agents as a state machine. Nodes are functions, edges are transitions, and state is checkpointed after every node via MemorySaver, SqliteSaver, or PostgresSaver. The only option here with first-class crash recovery: a workflow that dies at step 7 resumes from step 6 rather than restarting.
Claude Agent SDK
Subagents defined on options.agents, each with its own context window and its own tool list. That per-subagent window is the isolation boundary, so a worker cannot pollute the parent's context. Subagent messages carry parent_tool_use_id for per-agent cost attribution.
OpenAI Agents SDK
Coordination through handoffs: agent A transfers control to agent B via a typed tool call that passes conversation history along. No shared state bus. Guardrails run in parallel with execution and halt the run mid-generation on failure. Linear and branching chains, not arbitrary graphs.
CrewAI
Role-based crews. You define agents by role and goal, assemble them into a crew with tasks, and the framework resolves task dependencies and execution order. Native MCP support via crewai-tools[mcp] and A2A delegation between crews. The role-playing layer costs extra LLM calls.
Google ADK
Hierarchical agent trees where a parent delegates to children. to_a2a() generates Agent Cards so agents outside your system can discover and call yours. Four language SDKs (Python, TypeScript, Java, Go), which is the widest here and the reason multi-language shops pick it.
Microsoft Agent Framework
1.0 went GA on April 3, 2026, merging AutoGen and Semantic Kernel. Ships named orchestration patterns as primitives: sequential, concurrent, handoff, group chat, and Magentic-One, plus graph workflows. Group chat is the strongest human-in-the-loop story and the most expensive to run.
Every option above coordinates agents. None of them reduces what the coordination costs. The token bill is a function of how much context moves between agents and which model reads it, and that is a layer below the orchestrator. Picking LangGraph over CrewAI changes your debugging experience. It does not change your invoice.
What Orchestration Costs
Anthropic put a number on it: multi-agent systems use roughly 15 times more tokens than chat interactions, and single agents about 4 times more. They also found token usage alone explained 80% of performance variance on their BrowseComp evaluation. The cost is not an implementation detail you optimize away later. It is the mechanism.
The volume comes from context, not from agent count. Every delegation re-sends state. Every agent that reads a file keeps that file in its window for the rest of its run. Cognition, building their own search model, found their agent trajectories were "often spending >60% of their first turn just retrieving context." Most of what an orchestrated system pays for is agents reading things that turn out not to matter.
Three levers actually move the number.
Compress the context that moves between agents
Handoffs carry accumulated history. Morph Compact shrinks that payload 50 to 70% while keeping surviving sentences verbatim, which matters because a lossy summary breaks the next agent's ability to quote code exactly. It runs at 33,000 tok/s, so a compression step that would take about 90 seconds on a general model finishes in roughly 2.5 seconds. Pricing is $0.20 per 1M input tokens and $0.50 per 1M output.
Route each step to the cheapest model that can do it
A planner decomposing a task and a worker running a fixed extraction do not need the same model. On Morph, morph-dsv4flash is $0.139 per 1M input and $0.278 per 1M output; morph-glm52-744b is $1.10 and $4.10. That is roughly 8x on input and 15x on output between the tier you want planning on and the tier a mechanical subtask runs fine on. Routing a fan-out's workers down one tier is usually the single largest line-item change available. If you would rather not hand-assign, auto selects per request and bills $0.005 per routing decision.
Offload search so the planner never reads dead ends
The counterintuitive result is that adding an agent makes the system cheaper. WarpGrep explores in its own context and returns file spans instead of files, so the expensive model never pays to read the rejected paths. Applying the resulting edits is its own offload: morph-v3-fast merges edits at 10,500 tok/s instead of having the frontier model regenerate whole files.
| Cost driver | Why it grows | Lever |
|---|---|---|
| Context passed on handoff | Every delegation re-sends accumulated state | Compress to 30-50% of original, verbatim |
| Model tier on every step | Workers inherit the planner's model by default | Route mechanical steps down a tier |
| Exploration the planner reads | Rejected files stay in the window all run | Search subagent returns spans, not files |
When Not to Orchestrate
Single agents are the correct default, and most teams reach for orchestration a step or two before they should. The 90.2% result came from a research task that decomposes cleanly into independent searches. That is close to the best case for multi-agent, not the median case.
- Subtasks are genuinely independent and can be verified separately
- Subtask exploration would fill the main agent's context with material it does not need
- Wall-clock latency matters and the work parallelizes
- Different steps want genuinely different models or tool permissions
- Subtasks share mutable state and need to see each other's intermediate work
- The decomposition is not knowable before the task starts
- Total latency matters more than per-step quality, since planning adds a round trip
- You cannot yet attribute cost or failures per agent, which makes regressions unfindable
The failure mode worth naming: if agents need each other's intermediate work to be correct, the subtasks are not independent, and orchestration has converted a prompting problem into a distributed systems problem. You now own partial failure, ordering, and state reconciliation, and you own them inside a system whose components are nondeterministic. That is a much harder problem than the one you started with.
The honest sequence is to make a single agent work first, measure where its context actually goes, and split out the specific stage that is drowning it. In coding agents that stage is almost always search, which is why search is the first thing worth offloading and often the only thing that needs to be.
Before building an orchestrator, try a single agent with one specialized subagent for search. It captures most of the context-isolation benefit, keeps one control flow to debug, and does not require you to solve merge semantics or partial failure. If that does not close the gap, then the coordination layer is the real problem and worth building.
FAQ
What is AI agent orchestration?
AI agent orchestration is the runtime coordination of multiple AI agents working on a single task: deciding which agent runs, in what order, on what context, and what happens when one fails. The orchestrator owns control flow and context routing rather than doing the work itself.
What is the difference between agent orchestration and an agent framework?
A framework is a development-time concern, the library you build an agent with. Orchestration is a runtime concern, the coordination that happens when several agents run together. Some libraries do both, LangGraph most obviously. The distinction still earns its keep during debugging: framework problems appear at write time, orchestration problems appear under load.
What are the main AI agent orchestration patterns?
Planner-worker, pipeline, and parallel fan-out. Planner-worker decomposes at runtime and delegates to isolated workers. Pipeline runs a fixed sequence of transforming stages. Fan-out dispatches independent agents simultaneously and merges results. Most real systems compose two of the three.
Does multi-agent orchestration actually improve results?
Anthropic reported 90.2% improvement over single-agent Claude Opus 4 on their internal research eval, using Opus 4 as lead with Sonnet 4 subagents. The same write-up reports roughly 15x the token consumption of a chat interaction, and that token usage alone explained 80% of performance variance on BrowseComp. The gain is real and it is bought with tokens.
What does AI agent orchestration cost?
Cost tracks context volume rather than agent count, because every delegation re-sends state and every file an agent reads stays in its window. Cognition measured over 60% of an agent's first turn going to context retrieval. Compression, model routing per step, and offloading search to a subagent are the three levers that move the bill.
When should you not orchestrate agents?
When subtasks share mutable state, when the decomposition is not knowable in advance, or when end-to-end latency matters more than per-step quality. If agents need to see each other's intermediate work, they are not independent, and you have traded a prompting problem for a distributed systems problem.
Cut what orchestration costs to run
Compact compresses handoff context 50-70% at 33,000 tok/s. WarpGrep returns file spans instead of files. Both work with any orchestrator.