Malhar Ujawane

Blog

Explainer·(updated October 31, 2026)

Orchestrator-worker pattern: how AI agents split up work

The orchestrator-worker pattern has one LLM break a task into subtasks at runtime, run workers in parallel and merge the results. How it works, with Python.

Orchestrator-worker pattern: how AI agents split up work

The orchestrator-worker pattern is an agent design where a central LLM reads the task, decides at runtime which subtasks it needs, hands each one to a worker LLM with its own context, and then combines the workers’ results into one answer. It’s the pattern behind most deep research products and most coding agents that spawn helpers. What separates it from plain parallelization is that the subtasks are not fixed in code: the orchestrator invents them for each input.

How does the orchestrator-worker pattern work?

Anthropic’s Building effective agents (December 19, 2024) gives the definition: “a central LLM dynamically breaks down tasks, delegates them to worker LLMs, and synthesizes their results.” It then names the one property that matters. Compared with parallelization, the key difference is that “subtasks aren’t pre-defined, but determined by the orchestrator based on the specific input.” The examples it gives are coding changes that touch an unknown number of files, and search tasks that gather information from several sources.

The run has three phases:

  1. Plan. The orchestrator reads the request and writes a list of subtasks, each with an objective and an expected output.
  2. Delegate. Workers run, usually in parallel. Each starts with a clean context containing only its brief. A worker can be a single LLM call or a full subagent with tools.
  3. Synthesize. The orchestrator reads the workers’ short results and produces the answer. It may decide the results have gaps and run another round.

Diagram of the orchestrator-worker pattern. A request enters an orchestrator box, which sends three briefs in orange to three worker boxes, each with its own context. Each worker returns a short result to a synthesis step, which can loop back to the orchestrator for another round or produce the answer.

How is it different from parallelization and routing?

All three are in Anthropic’s list of workflow patterns, and they’re easy to confuse.

  • Parallelization runs subtasks you defined in code at the same time, such as one call screening for policy violations while another answers. The split is fixed.
  • Routing classifies an input and sends it down one of several fixed paths. One input, one path. I cover a cost-driven version in LLM routing.
  • Orchestrator-workers lets the model choose how many subtasks, and what they are, for each input.

That last difference is why I’d file orchestrator-workers between workflows and agents in workflows vs agents. The outer structure is fixed code (plan, fan out, merge). The content of the plan is the model’s decision.

What does the pattern look like in code?

Here’s a minimal version with real parallelism. The orchestrator, workers and synthesizer are stubs that sleep to stand in for model latency, so this runs offline. The orchestrator picks documents by overlap with the question, a crude stand-in for an LLM’s judgment that will turn out to be instructive.

# orchestrator.py: orchestrator-workers with real parallelism, stub LLM calls.
# Python 3 stdlib only. time.sleep stands in for model latency.
import time
from concurrent.futures import ThreadPoolExecutor

DOCS = {
    "pricing.md":  "Pro plan is $20 a month. Annual billing saves two months.",
    "limits.md":   "Free plan allows 50 requests a day. Pro allows 5,000.",
    "security.md": "Data is encrypted at rest. SSO is on the Enterprise plan.",
    "changelog.md": "v2.3 added webhooks. v2.4 fixed CSV export.",
}

def orchestrator_plan(question):
    """Stub LLM: decides the subtasks from the input. Not a fixed list."""
    time.sleep(0.4)
    words = set(question.lower().replace("?", "").split())
    picks = [d for d, text in DOCS.items()
             if words & set(text.lower().replace(".", "").replace(",", "").split())]
    return [{"doc": d, "objective": f"Extract facts relevant to: {question}",
             "output": "one sentence"} for d in picks]

def worker(task):
    """Stub LLM worker: clean context, one bounded job, short return."""
    time.sleep(1.0)
    return f"[{task['doc']}] {DOCS[task['doc']]}"

def synthesise(question, findings):
    time.sleep(0.4)
    return f"Q: {question}\n" + "\n".join("  " + f for f in findings)

question = "What does the Pro plan cost and what are its request limits?"
for mode in ("sequential", "parallel"):
    t0 = time.perf_counter()
    tasks = orchestrator_plan(question)
    if mode == "sequential":
        findings = [worker(t) for t in tasks]
    else:
        with ThreadPoolExecutor(max_workers=len(tasks)) as pool:
            findings = list(pool.map(worker, tasks))
    answer = synthesise(question, findings)
    print(f"{mode}: {len(tasks)} workers, {time.perf_counter() - t0:.1f}s")
print(answer)

Output (Python 3.13.11):

sequential: 3 workers, 3.8s
parallel: 3 workers, 1.8s
Q: What does the Pro plan cost and what are its request limits?
  [pricing.md] Pro plan is $20 a month. Annual billing saves two months.
  [limits.md] Free plan allows 50 requests a day. Pro allows 5,000.
  [security.md] Data is encrypted at rest. SSO is on the Enterprise plan.

Two lessons in eight lines. First, parallel workers cut wall-clock time roughly to the plan plus the slowest worker plus synthesis. Anthropic reports the same effect at scale: running subagents in parallel, and having each use tools in parallel, cut research time “by up to 90% for complex queries” in their research system (June 13, 2025).

Second, look at security.md. The orchestrator sent a worker there because the word “plan” matched. That worker cost a full call and added noise the synthesizer has to ignore. A real LLM orchestrator makes the same kind of mistake with more fluent reasons. Anthropic’s early versions spawned “50 subagents for simple queries.” Over-delegation is the default failure of this pattern, and the fix lives in the orchestrator’s instructions.

How do I write a good orchestrator?

Anthropic’s research post is the most detailed public guide, and most of its advice is about the orchestrator’s prompt.

Brief workers completely. “Each subagent needs an objective, an output format, guidance on the tools and sources to use, and clear task boundaries.” Without that, the post says, agents duplicate work, leave gaps or miss what they were sent to find. Their example of a bad brief is a one-line “research the semiconductor shortage”, which led subagents to repeat each other’s searches.

Scale effort to the query. They embedded rules in the prompt: simple fact-finding needs one agent with 3 to 10 tool calls; direct comparisons might need 2 to 4 subagents with 10 to 15 calls each; complex research might use more than 10 subagents.

Return summaries, not transcripts. Anthropic’s context engineering post (September 29, 2025) says a sub-agent “returns only a condensed, distilled summary of its work (often 1,000-2,000 tokens).” The orchestrator’s context is the bottleneck; protect it.

Save the plan outside the context. Their lead agent writes its plan to memory, because if its context window passes 200,000 tokens it gets truncated, and the plan is what it needs to keep.

Separate synthesis from attribution if you need citations. Their system passes findings to a dedicated CitationAgent that matches claims to sources after the report is written.

Where does the orchestrator-worker pattern break?

Coupled subtasks. Workers can’t see each other. If subtask B depends on a choice made in subtask A, parallel workers will make conflicting choices. Walden Yan’s Don’t Build Multi-Agents (June 12, 2025) is built on this failure. Split only along seams where workers truly don’t need each other’s decisions.

The synchronous wall. Anthropic’s lead agent waits for each batch of subagents to finish, which “simplifies coordination, but creates bottlenecks.” One slow worker holds the run, and the orchestrator can’t redirect a worker that’s going the wrong way.

Token cost. Anthropic puts multi-agent systems at about 15 times the tokens of a chat. Every worker pays for its own prompt and its own exploration.

Lossy merges. The synthesizer only sees summaries. Facts a worker judged unimportant are gone. Ask workers for exact quotes and source locations alongside their conclusions.

What I would do

  1. Use the pattern when the number and shape of subtasks depend on the input. If they don’t, use fixed parallelization in code and save the planning call.
  2. Make the plan structured output: a JSON list of subtasks with objective, sources, output format and a call budget each. Validate it before fanning out, including a cap on the number of workers.
  3. Run workers with a thread pool or async calls, a per-worker timeout and a per-worker token limit.
  4. Have workers return short structured findings with citations. Keep raw documents out of the orchestrator.
  5. Add one evaluation step after synthesis, the evaluator-optimizer idea, that checks the answer against the original question and triggers one more round only if something specific is missing.

Deep research agents are this pattern plus search tools and a report writer, and multi-agent systems covers the wider family.

FAQ

What is the orchestrator-worker pattern in AI?

A design where one LLM breaks a task into subtasks at runtime, delegates each to a worker LLM, and combines the results. Anthropic lists it among its core agentic workflow patterns.

What is the difference between orchestrator-worker and parallelization?

In parallelization, the subtasks are defined in your code. In orchestrator-workers, the orchestrator model decides the subtasks for each input, so the number and content of workers varies.

Is the orchestrator-worker pattern a multi-agent system?

When the workers are full agents with their own tools and context windows, yes, and it’s the most common multi-agent shape. When workers are single LLM calls, it’s closer to a dynamic workflow.

Should workers talk to each other?

Usually not. Keeping workers independent is what makes them safe to run in parallel. If they need to share decisions, the task is probably sequential and belongs in one agent.

Sources

Get new posts in your inbox

© 2026 Malhar Ujawane. Built with Astro.