What is a multi-agent system? How AI agents work together
A multi-agent system splits one task across several LLM agents with separate contexts. When it beats one agent, what it costs, and why handoffs fail.

A multi-agent system is an AI application where several language model agents, each with its own instructions, tools and context window, work on parts of one task and pass results between them. The common shape is a lead agent that splits the work, hands pieces to other agents, and combines what comes back. It can beat a single agent on wide, parallel work, and it costs several times more tokens to do so.
What counts as a multi-agent system?
An AI agent is a model calling tools in a loop. A multi-agent system has more than one of those loops, and the loops talk to each other. The thing that makes them separate agents, rather than one agent with many tools, is separate context: each agent sees only what it was given and what it found.
Three shapes come up again and again:
- Orchestrator and workers. A lead agent plans, spawns workers for subtasks and synthesizes the results. Workers don’t talk to each other. This is the pattern in orchestrator-worker agents and the one most production systems use.
- Handoffs. A triage agent routes the conversation to a specialist, and the specialist takes over. OpenAI’s Agents SDK docs describe this next to “agents as tools”, where a manager keeps control and calls specialists like functions.
- Conversation or role-play. Agents with roles (planner, coder, reviewer) exchange messages. Microsoft’s AutoGen (Wu et al., August 2023) built a framework around “multiple agents that can converse with each other,” and MetaGPT (Hong et al., August 2023) encoded standard operating procedures into prompts so role agents work like an assembly line.
Across organizational boundaries, agents built by different teams need a wire protocol to find and call each other. That’s what the A2A protocol standardizes.
Does a multi-agent system beat a single agent?
On the right task, yes, by a lot. Anthropic’s write-up of its research system (June 13, 2025) is the best public evidence. A setup with Claude Opus 4 as lead and Claude Sonnet 4 subagents “outperformed single-agent Claude Opus 4 by 90.2%” on Anthropic’s internal research eval.
The explanation they give is more useful than the headline. On the BrowseComp benchmark, token usage alone explained 80% of the variance in performance, and token usage plus number of tool calls plus model choice explained 95%. Multi-agent works, in their words, largely because it spends enough tokens to solve the problem: each subagent gets a fresh context window to search in, so the system as a whole can read far more than one window holds.
The same post states the bill. Agents use about 4 times the tokens of a chat interaction, and multi-agent systems about 15 times. It also names where the design is a poor fit: domains where agents need shared context or have many dependencies, and coding, because “most coding tasks involve fewer truly parallelizable tasks than research.”
Why do multi-agent systems fail?
Because every handoff loses context. Walden Yan of Cognition put this most sharply in Don’t Build Multi-Agents (June 12, 2025), with two principles:
- “Share context, and share full agent traces, not just individual messages”
- “Actions carry implicit decisions, and conflicting decisions carry bad results”
His example: ask for a Flappy Bird clone, split it into “background with pipes” and “the bird”, and one subagent builds a Super Mario style background while the other builds a bird that doesn’t match it. Each made a reasonable choice. The choices conflicted, and the agent that merges them has to reconcile work it didn’t see being made.
A research group that includes Ion Stoica, Matei Zaharia and Joseph Gonzalez measured this at scale. Why Do Multi-Agent LLM Systems Fail? (Cemri et al., March 2025) annotated more than 1,600 traces across 7 popular multi-agent frameworks and built a taxonomy of 14 failure modes in three groups: system design issues, inter-agent misalignment, and task verification. Their opening line is the summary: performance gains on popular benchmarks “are often minimal.”
What do multi-agent systems cost?
Here is the arithmetic, using Anthropic’s multipliers and a deliberately round price. The second half shows why handoffs hurt: if each one has some chance of dropping or distorting the task, those chances multiply.
# multi_agent_math.py: what a multi-agent system costs and how handoffs compound.
# Python 3 stdlib only. Multipliers from Anthropic's June 2025 research-system post:
# agents use ~4x the tokens of a chat, multi-agent systems ~15x.
CHAT_TOKENS = 5_000 # assumption: one chat answer, input plus output
PRICE_PER_M = 10.00 # assumption: blended USD per 1M tokens, not a real price
print("architecture tokens cost/run cost/10k runs")
for name, mult in [("chat", 1), ("single agent", 4), ("multi-agent", 15)]:
tokens = CHAT_TOKENS * mult
usd = tokens * PRICE_PER_M / 1e6
print(f"{name:<14} {tokens:>7,} ${usd:>6.3f} ${usd * 10_000:>9,.0f}")
print("\nP(run succeeds) when every handoff must keep the task intact")
print("handoffs p=0.99 p=0.95 p=0.90")
for k in (1, 3, 5, 10):
print(f"{k:>8} " + " ".join(f"{p ** k:6.1%}" for p in (0.99, 0.95, 0.90)))
Output (Python 3.13.11):
architecture tokens cost/run cost/10k runs
chat 5,000 $ 0.050 $ 500
single agent 20,000 $ 0.200 $ 2,000
multi-agent 75,000 $ 0.750 $ 7,500
P(run succeeds) when every handoff must keep the task intact
handoffs p=0.99 p=0.95 p=0.90
1 99.0% 95.0% 90.0%
3 97.0% 85.7% 72.9%
5 95.1% 77.4% 59.0%
10 90.4% 59.9% 34.9%
The chat size and the $10 blended price are my assumptions; swap in your own. The multipliers are Anthropic’s. The second table is a simplification (handoff failures aren’t independent in real systems), but it captures the intuition behind Cognition’s argument: ten handoffs at 95% each leave you with a coin flip. Orchestrator-worker designs keep the number of handoffs per worker at one, which is part of why they hold up better than chains of agents passing work along.
When should I use a multi-agent system?
I use three tests.
Is the work wide? Research across many sources, comparing many documents, auditing many repositories: tasks that split into independent threads. Anthropic’s lead agent scales from one agent with 3 to 10 tool calls for simple fact-finding to “more than 10 subagents with clearly divided responsibilities” for complex research.
Does the value cover 15x the tokens? A research report someone would pay an analyst for, yes. A support answer at volume, rarely.
Can workers succeed without seeing each other? If worker B needs worker A’s decisions, you have a sequential task, and one agent with good context management is safer. Subagents that only read and summarize are the low-risk middle ground: the main agent keeps every decision, and the helpers keep the noise out of its window.
rSearch, the search product I built, runs a reasoning model over search results. Looking at a design like that, the line I’d draw is between fetching and deciding. Fetching parallelizes well across agents; deciding what the answer is should stay in one place.
Where do multi-agent systems break in production?
Anthropic’s post is candid about early failures: agents “spawning 50 subagents for simple queries,” searching endlessly for sources that don’t exist, and distracting each other with too many updates. Their fixes were prompt-level: teach the lead agent how to delegate (objective, output format, tool guidance, task boundaries) and scale effort to the query.
Two structural limits are worth knowing. Their lead agent runs subagents synchronously, waiting for each batch to finish, so one slow subagent stalls the run and the lead can’t steer mid-flight. And because agents are stateful and long-running, they deploy updates gradually with rainbow deployments so running agents aren’t broken by a code change.
For coding, where parallel agents are tempting, the practical answer has been isolation rather than coordination: give each agent its own branch and working copy, as in git worktrees for parallel agents, and let a human merge.
What I would do
- Build the single-agent version first and measure where it fails. If it fails because the context fills up with search results, try read-only subagents before a full multi-agent design.
- When you split, use orchestrator-worker with one handoff per worker. Give each worker an objective, an output format, the tools it may use and a stop condition, in writing.
- Return short structured results from workers and keep raw material out of the lead’s context.
- Log full traces for every agent. The MAST taxonomy is a good checklist when reading them: was the failure in the design, in the handoff, or in verification?
- Price it at 15x a chat before you build it, then check the real token logs after.
FAQ
What is an example of a multi-agent system?
Anthropic’s Research feature: a lead agent plans the research, spawns subagents that search in parallel, and a separate citation agent attributes claims to sources. Coding tools that spawn read-only explorer agents are a smaller example.
What is the difference between a single agent and a multi-agent system?
A single agent is one model loop with one context window. A multi-agent system has several loops with separate context windows that pass results to each other, which buys parallelism and more total context at the cost of tokens and coordination.
Are multi-agent systems worth it?
For broad research tasks, Anthropic reports a 90.2% gain over a single agent on its internal eval at roughly 15 times the tokens of a chat. For tightly coupled work like most coding, Cognition and the MAST study both argue the coordination cost usually outweighs the gain.
What frameworks are used to build multi-agent systems?
AutoGen, MetaGPT and the OpenAI Agents SDK are three with public papers or docs. You can also build the orchestrator-worker pattern directly on a model API with a thread pool, which I’d do first to see what the model sees.
Sources
- Anthropic, How we built our multi-agent research system, June 13, 2025
- Walden Yan, Cognition, Don’t Build Multi-Agents, June 12, 2025
- Cemri et al., Why Do Multi-Agent LLM Systems Fail?, arXiv, March 2025
- Wu et al., AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation, arXiv, August 2023
- Hong et al., MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework, arXiv, August 2023
- OpenAI Agents SDK, Agent orchestration (read 2026-10-31)