Evaluator-optimizer pattern: reflection loops for AI agents
The evaluator-optimizer pattern pairs a generator LLM with an evaluator that sends feedback in a loop until the output passes. When it works, with Python.

The evaluator-optimizer pattern is an AI workflow where one LLM call produces an answer, a second step evaluates it against explicit criteria and returns feedback, and the generator revises until the evaluator passes it or a round limit is hit. It’s also called a reflection loop or self-refinement. It works when the evaluator can tell good from bad more reliably than the generator can get it right first time, which is why the strongest version uses an evaluator that runs real checks rather than one that just asks a model for its opinion.
How does the evaluator-optimizer pattern work?
Anthropic’s Building effective agents (December 19, 2024) describes it in one line: “one LLM call generates a response while another provides evaluation and feedback in a loop.” The post gives the condition for using it: “when we have clear evaluation criteria, and when iterative refinement provides measurable value.” Its two signs of a good fit are that a response gets better when a human articulates feedback, and that an LLM can produce that kind of feedback.
The loop has four parts:
- Generator: writes a draft from the task and any feedback so far.
- Evaluator: scores the draft against criteria and, if it fails, says specifically what’s wrong.
- Feedback channel: the evaluator’s notes go back into the generator’s next prompt.
- Stop rule: pass, or a maximum number of rounds.
Anthropic’s examples are literary translation, where an evaluator model catches nuance the translator missed, and complex search, where the evaluator decides whether another round of searching is warranted. That second example is the inner loop of most deep research agents.
What does the research say about self-correction?
There are three papers I’d read, and they pull in different directions.
Self-Refine (Madaan et al., March 2023) used a single LLM as generator, feedback provider and refiner, with no extra training. Across 7 tasks from dialog responses to math reasoning, with GPT-3.5, ChatGPT and GPT-4, outputs improved by about 20% absolute on average over one-shot generation, judged by humans and automatic metrics.
Reflexion (Shinn et al., March 2023) moved the idea into agents. After a failed attempt, the agent writes a verbal reflection on what went wrong into an episodic memory buffer and reads it on the next try. On HumanEval it reported 91% pass@1, against 80% for GPT-4 at the time. The feedback signal there included test results, which matters for what follows.
Large Language Models Cannot Self-Correct Reasoning Yet (Huang et al., October 2023, ICLR 2024) is the counterweight. It studied intrinsic self-correction, where the model critiques itself with no external feedback, and found that on reasoning tasks models “struggle to self-correct their responses without external feedback, and at times, their performance even degrades.”
Put together, my reading is this: a loop helps when the evaluator brings information the generator didn’t have. That can be a test result, a compiler error, a retrieved document, a stricter rubric or a stronger model. A model re-reading its own reasoning with nothing new tends to talk itself into or out of answers at random.
What does an evaluator-optimizer loop look like in code?
This loop writes a slugify function. The generator is a stub that picks the next draft based on the feedback, standing in for an LLM revising its code. The evaluator is real: it executes the draft against test cases and reports the first failure in specific terms.
# eval_opt.py: an evaluator-optimizer loop where the evaluator runs real tests.
# Python 3 stdlib only. The generator is a stub that "revises" from feedback,
# standing in for an LLM; the evaluator is real code, which is the point.
import re
CANDIDATES = { # what a model might write on each revision
"v1": "def slugify(s):\n return s.lower().replace(' ', '-')",
"v2": "def slugify(s):\n s = re.sub(r'[^a-z0-9 ]', '', s.lower())\n return s.replace(' ', '-')",
"v3": "def slugify(s):\n s = re.sub(r'[^a-z0-9]+', '-', s.lower())\n return s.strip('-')",
}
TESTS = [("Hello World", "hello-world"), ("What is RAG?", "what-is-rag"),
(" MCP vs A2A ", "mcp-vs-a2a"), ("C'est la vie!", "c-est-la-vie")]
def generator(feedback):
"""Stub LLM: picks the next draft based on what the evaluator complained about."""
if feedback is None:
return "v1"
if "'?'" in feedback or "'!'" in feedback:
return "v2"
return "v3"
def evaluator(code):
"""Real check: run the draft against test cases, return pass or concrete feedback."""
ns = {"re": re}
exec(code, ns)
fails = [(i, ns["slugify"](i), want) for i, want in TESTS if ns["slugify"](i) != want]
if not fails:
return True, "all tests pass"
i, got, want = fails[0]
bad = sorted(set(got) - set("abcdefghijklmnopqrstuvwxyz0123456789-"))
hint = f"bad chars {bad}" if bad else "repeated or edge hyphens"
return False, f"{len(fails)}/{len(TESTS)} fail; {i!r} -> {got!r}, want {want!r}; {hint}"
feedback, MAX_ROUNDS = None, 4
for rnd in range(1, MAX_ROUNDS + 1):
draft = generator(feedback)
ok, feedback = evaluator(CANDIDATES[draft])
print(f"round {rnd}: {draft} -> {feedback}")
if ok:
break
else:
print("gave up after max rounds")
Output (Python 3.13.11):
round 1: v1 -> 3/4 fail; 'What is RAG?' -> 'what-is-rag?', want 'what-is-rag'; bad chars ['?']
round 2: v2 -> 2/4 fail; ' MCP vs A2A ' -> '--mcp--vs--a2a-', want 'mcp-vs-a2a'; repeated or edge hyphens
round 3: v3 -> all tests pass
The interesting line is round 2. The fix for punctuation made the failure count drop from 3 to 2 but exposed a different bug, doubled hyphens from repeated spaces. A model-only evaluator reading v2 could easily have called it “looks correct”. The test runner can’t be talked into that. Note also that the feedback is a specific input, output and expected value, not a grade. Feedback the generator can act on is what makes the next round better.
Never exec model-written code outside a sandbox in a real system. It’s fine here because the candidates are hard-coded.
When should I use an LLM as the evaluator?
When there’s no cheaper, harder check. Translation quality, tone, whether a summary covers a source, whether a research answer addresses every part of a question: these have no unit test, so an LLM-as-a-judge with a written rubric is the practical evaluator.
Three things make a model evaluator worth its tokens:
- Different context. Give the evaluator the criteria and the source material, not the generator’s reasoning. It should judge the output, not be persuaded by the argument for it.
- A specific rubric. “Is this good?” produces vague feedback. “List every claim not supported by the attached source” produces feedback the generator can fix. This is also a direct defense against hallucinations in summaries.
- A stronger or different model when you can afford it. A model grading itself shares its own blind spots.
Anthropic’s research system post (June 13, 2025) used an LLM judge for its own evals: a single call, a single prompt, scores from 0.0 to 1.0 and a pass-fail grade, against criteria like factual accuracy, citation accuracy and completeness. The same shape works as the evaluator inside a loop.
Where does the evaluator-optimizer pattern break?
No external signal. Huang et al. is the warning. If the evaluator is the same model with the same information, expect small or negative gains on reasoning tasks.
Oscillation. The generator fixes issue A and reintroduces issue B, then flips back. Pass the full feedback history, not only the latest note, and cap rounds.
A persuadable evaluator. A model evaluator that reads a revision framed as “fixed the issue” can take the framing at face value. Require it to cite the specific criterion each failure violates and the evidence for each pass, and spot-check passes by hand.
Cost. Every round is at least two calls with a growing prompt. Three rounds can triple the cost of a request. Measure how often round 2 and round 3 actually change the outcome; if they rarely do, cut them.
What I would do
- Look for a hard check first: tests, a schema validator, a linter, a SQL
EXPLAIN, a citation matcher. If one exists, it’s your evaluator, and the loop is almost free to justify. - Where judgment is needed, write the rubric as a checklist and have the evaluator return failures as structured output, one item per violated criterion.
- Cap at two or three rounds and return the best draft with its evaluation if it never passes. Don’t silently ship a failing draft.
- Log every round. The logs become your test set for tuning the generator prompt, which often removes the need for the loop.
- Treat it as one component. It fits after synthesis in an orchestrator-worker system, or around a single step in a workflow, and the agent architectures post shows where it slots into the rest.
FAQ
What is the evaluator-optimizer pattern?
A loop where one LLM generates a response and a second step evaluates it and sends feedback, repeating until the response meets the criteria or a round limit is reached. Anthropic names it as one of five core workflow patterns.
Is a reflection loop the same as evaluator-optimizer?
Mostly. Reflection usually means the same model critiques its own output, as in Self-Refine, or writes lessons for its next attempt, as in Reflexion. Evaluator-optimizer is the general shape, and the evaluator can be another model, a rubric or real code.
Can LLMs correct their own mistakes?
With external feedback such as test results, retrieved evidence or tool errors, often yes. Without it, Huang et al. found models struggle to self-correct reasoning and sometimes get worse.
How many refinement rounds should I run?
Start with a cap of two or three and measure. Self-Refine repeated feedback and refinement for a set number of iterations or until the model said no more refinement was needed; in production, the useful number is whatever your logs show still changes the outcome.
Sources
- Anthropic, Building effective agents, December 19, 2024
- Anthropic, How we built our multi-agent research system, June 13, 2025
- Madaan et al., Self-Refine: Iterative Refinement with Self-Feedback, arXiv, March 2023
- Shinn et al., Reflexion: Language Agents with Verbal Reinforcement Learning, arXiv, March 2023
- Huang et al., Large Language Models Cannot Self-Correct Reasoning Yet, arXiv, October 2023 (ICLR 2024)