Malhar Ujawane

Blog

Explainer·(updated October 31, 2026)

Workflows vs agents: which AI architecture to pick

AI workflows follow code paths you define; agents let the model choose each step. The five workflow patterns, when an agent is worth it, and a cost simulation.

Workflows vs agents: which AI architecture to pick

An AI workflow runs language model calls along code paths you wrote in advance; an AI agent lets the model decide the next step, including which tool to call and when to stop. Anthropic’s Building effective agents (December 19, 2024) draws the line exactly there: workflows are “orchestrated through predefined code paths”, while in agents “LLMs dynamically direct their own processes and tool usage.” Pick a workflow when you can write the steps down, and an agent only when you can’t.

Why does the workflow vs agent distinction matter?

Because it decides what you can predict. With a workflow, you know how many model calls a request makes, what each one sees, and where to look when the output is wrong. With an agent, the number of calls, the cost, the latency and the path are all decided at run time by the model.

Both sides of that trade are real. Anthropic’s post says agents fit “open-ended problems where it’s difficult or impossible to predict the required number of steps”, and also that “the autonomous nature of agents means higher costs, and the potential for compounding errors.” Its first recommendation is to find “the simplest solution possible, and only increasing complexity when needed”, which often means no loop at all: one model call with good retrieval and examples.

What are the five workflow patterns?

The same post names five patterns, and between them they cover most production LLM features I’d expect to meet:

  1. Prompt chaining. “Each LLM call processes the output of the previous one.” Write an outline, check it in code, then write the draft. Use it when the task splits cleanly into fixed steps.
  2. Routing. “Routing classifies an input and directs it to a specialized followup task.” Send refunds to one prompt and technical questions to another, or easy queries to a cheap model, which is the idea behind LLM routing.
  3. Parallelization. Several calls run at once and code combines the results, either by splitting a task into sections or by voting across repeated attempts.
  4. Orchestrator-workers. “A central LLM dynamically breaks down tasks, delegates them to worker LLMs, and synthesizes their results.” The subtasks aren’t known in advance, so this one is already halfway to an agent. More in the orchestrator-worker pattern.
  5. Evaluator-optimizer. One call generates, another evaluates and gives feedback, in a loop. Covered in the evaluator-optimizer pattern.

Notice that only the last two give the model any say over the control flow, and even there your code sets the shape. In a full agent, the model owns the loop.

When should you use an agent instead of a workflow?

OpenAI’s practical guide to building agents gives three signals that a task has resisted ordinary automation and might suit an agent:

  • Complex decision-making: judgment calls, exceptions and context-sensitive decisions, such as refund approval.
  • Difficult-to-maintain rules: rule sets that have grown so large that every change is costly and error-prone.
  • Heavy reliance on unstructured data: reading documents, interpreting natural language, talking to users.

It adds a sentence I agree with: “Before committing to building an agent, validate that your use case can meet these criteria clearly. Otherwise, a deterministic solution may suffice.”

My own test is shorter. Can I draw the flowchart? If yes, build the flowchart and put model calls in the boxes. If the flowchart would need a box labelled “figure out what to do next” that loops back on itself, that box is an agent, and it should be as small as possible.

How much more does an agent cost to run?

The bill depends on how many calls a run makes, and with an agent that number is a distribution, not a constant. This simulation compares a three-step workflow with an agent that keeps calling the model until a step lands it on the answer, capped at 15 calls. The per-step probabilities are made up to represent easy, mixed and hard tasks; the shape is the point, not the numbers.

# workflow_vs_agent.py: what "the model decides the path" does to your call count.
# Python 3 stdlib only. A simulation with made-up probabilities, not a benchmark:
# the workflow always makes 3 calls (extract -> draft -> check). The agent keeps
# calling the model until a step lands it on the answer, with a cap of 15 calls.
import random, statistics

def agent_calls(p_done_per_step, cap=15):
    for n in range(1, cap + 1):
        if random.random() < p_done_per_step:
            return n, True
    return cap, False

random.seed(42)
print(f"{'task mix':<26}{'mean':>6}{'p50':>5}{'p95':>5}{'max':>5}{'gave up':>9}")
print(f"{'workflow (fixed 3 calls)':<26}{3:>6.1f}{3:>5}{3:>5}{3:>5}{'0%':>9}")
for label, p in [("agent, easy tasks", 0.6), ("agent, mixed tasks", 0.3), ("agent, hard tasks", 0.12)]:
    runs = [agent_calls(p) for _ in range(10_000)]
    calls = sorted(n for n, _ in runs)
    failed = sum(not ok for _, ok in runs) / len(runs)
    p95 = calls[int(0.95 * len(calls)) - 1]
    print(f"{label:<26}{statistics.mean(calls):>6.1f}{statistics.median(calls):>5.0f}{p95:>5}{calls[-1]:>5}{failed:>9.0%}")

Output (Python 3.14.5):

task mix                    mean  p50  p95  max  gave up
workflow (fixed 3 calls)     3.0    3    3    3       0%
agent, easy tasks            1.7    1    4   12       0%
agent, mixed tasks           3.3    2    9   15       1%
agent, hard tasks            7.0    6   15   15      14%

Three things to read off this. On easy tasks the agent is cheaper than the workflow, because it stops as soon as it’s done, while the workflow always pays for three steps. On mixed tasks the mean is close to the workflow’s, but the p95 is three times higher, and p95 is what your latency SLO and your budget alarms see. On hard tasks the agent spends more and still gives up on 14% of runs at the cap. In a real agent each call also costs more than the last, because the history grows, which I put numbers on in what agentic AI is.

The workflow’s weakness doesn’t show in the table: when its three fixed steps don’t fit the input, it fails quietly instead of adapting. That is the case for agents.

Can you combine workflows and agents?

Yes, and most production systems I’d design do. The common shapes:

  • A workflow with an agentic step. A fixed pipeline (classify, retrieve, answer, check) where one box, say “investigate the failing test”, is an agent with a step cap.
  • An agent with workflow tools. The model decides what to do, but each tool is a deterministic pipeline: “generate invoice PDF” is one tool call, not twelve model decisions.
  • A router in front. A cheap classifier sends predictable requests to a workflow and open-ended ones to an agent.

OpenAI’s guide recommends the same incremental path: start with a single agent, add tools before adding agents, and move to multi-agent designs only when needed. Anthropic’s advice is to begin with the API directly rather than a framework, so you can see every prompt.

What I would do

Default to a workflow. Write the steps as plain functions, each with one model call, and check the output of each in code. Add a model-directed loop only for the step where the path depends on what the model finds, and give that loop a step cap, a token budget and a log.

Then measure, on your own tasks, three numbers for both designs: success rate, p95 calls per task and p95 latency. If the agent doesn’t win on success rate by enough to pay for its p95, keep the workflow. From years on data platforms, my bias is toward the system whose worst day I can predict; how to evaluate AI agents covers how to get those numbers.

FAQ

What is the difference between an AI workflow and an AI agent?

In a workflow, your code decides the sequence of model calls and tools. In an agent, the model decides the next action at each step, based on what the previous actions returned, until it finishes or hits a limit.

Are agents better than workflows?

Neither is better in general. Agents handle open-ended tasks where the steps can’t be predicted; workflows are cheaper, faster and easier to debug when they can. Anthropic recommends the simplest option that works.

Is RAG a workflow or an agent?

Classic RAG is a workflow: retrieve, then generate, every time. When the model decides whether, what and how many times to retrieve, it becomes agentic RAG.

What is prompt chaining?

A workflow pattern where a task is split into fixed steps and each model call works on the previous call’s output, often with a check in code between steps.

Sources

Get new posts in your inbox

© 2026 Malhar Ujawane. Built with Astro.