A coding agent that gets better on Tuesday because of how it performed on Monday sounds like a training run. In GEPA it is a diff to a text file, and the hard part is not the optimiser. It is the scoreboard you have to build first.

The Core Takeaway: GEPA replaces gradient descent with reflection over transcripts, which makes improvement cheap and reviewable. That also makes the evaluator load-bearing in a way RL hides: a bad reward signal now produces a confidently rewritten prompt instead of a bad checkpoint, and the prompt ships.

What GEPA actually does

The paper is GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning [1]. The name is Genetic-Pareto, and the pitch is that language is a richer learning medium than a scalar reward: a trajectory tells you why a step failed, and a gradient does not. The comparison is against group-relative policy optimisation and similar methods that need thousands of rollouts to learn a task.

The loop has four moves, and every one of them is designed to keep a change attributable.

Reflect on real transcripts, not on the prompt text alone. The reflection model is shown how the parent behaved, plus a note about what failed, and it rewrites from that evidence. This is the difference between asking a model to improve your instructions, which yields confident prose with no contact with behaviour, and asking it to improve this instruction given these failures.

Rewrite one component per round. Reef, the open-source implementation I read, runs the search over a whole harness tree of rules, skills and commands, and a single round changes one evolvable file [2]. One change at a time is what makes the improvement attributable.

Accept only on strict improvement against the parent, on the same problems. The child runs on exactly the batch the parent saw. No win, no adoption, and the served tree is unchanged.

Sample parents from a Pareto archive, not from the top of a leaderboard. This is the move that separates GEPA from hill climbing. Each candidate is weighted by how many individual problems it is the best at, so a candidate that is mediocre on average but strongest on one kind of problem still gets a turn as a parent [2]. Without that, the search converges on the first thing that worked and quietly loses the specialisation that made it useful.

Why the loop is affordable

Naive evolution is expensive because every candidate is measured on everything. Three mechanisms cut the bill, and they are worth knowing because they shape what the results mean.

MechanismEffectCost it removes
Small minibatch for the child testa failed child costs only those few runsfull validation on every mutation
Full validation only for winnersexpensive scoring is spent on survivorswasted measurement
Score cache keyed on tree and problema tree never re-runs a problem it already scored onrepeat evaluation
Free minibatch when the parent is the served treeits real traffic is already collected and scoredre-running production traffic

The archive stores more than prompts. Reef keeps lineage, per-problem scores, which problems a candidate currently leads, which component to rewrite next, and how much of the metric-call budget is spent, and it commits that state each step, so an interrupted search resumes rather than restarting [2].

That budget matters. The metric-call budget is the actual currency of the method, and a run that ignores it will happily spend a week of evaluation to move a number by a rounding error.

The part that decides everything

Every mechanism above protects you from a regression on the metric. None protects you from the metric being wrong.

Point GEPA at a proxy and it will maximise the proxy, one strict improvement at a time, and then publish the result. The strict-better rule gives the output a legitimacy it has not earned, because the comparison is laundered through a number you chose. This is the same shape as trusting a summary instead of reading the log: the loop makes the score go up, and the score is a claim, not evidence.

The paper’s own framing is honest about the price. Reflection needs transcripts, and transcripts need a task that produces checkable outcomes. Where outcomes are not checkable, there is nothing to reflect on and GEPA degrades into prompt golf.

A scoreboard you already own

The reason this is worth doing in a small workspace rather than only in a lab is that the missing artifact is often already present, disguised as tooling.

I run four linters over this repository that each return a per-file pass or fail and each have a documented set of failures they were written to catch: a citation linter that checks reference entries against in-text markers, a reference resolver that fetches every target and reports dead links, a secret and style scanner for public content, and a UI performance gate that checks named guards still exist. Every one of those is a scored task with ground truth, and every one of them has a history of real failures I fixed by hand.

That is a better GEPA corpus than a benchmark, for one reason: I know which answers are right, so a strict improvement on the held-out files is checkable rather than plausible. A benchmark tells you that a prompt scored 0.71. My linters tell you that the rewrite fixes four known failure classes and breaks none.

The same trap applies locally, though. Score against the files I already fixed and the optimiser will learn to fit those files. The discipline is the one the method itself uses: hold files back, accept only strict improvement on the held-out set, and treat a specialist that wins on one class as a parent worth keeping rather than an average worth discarding.

What I would not do yet

Three refusals, in order of how much they matter.

Do not train weights on live traffic in a private workspace. Reef’s model and test-time paths ingest requests and responses to learn from them [3]. In this repository that traffic is draft posts, knowledge-base notes and research material, all of which are private by rule. A local trainer over local files is defensible. Anything that ships telemetry is not, and “it is open source” does not change where the bytes go.

Do not treat a published version as a validated one. The project is at v0.1.3 with a very fast merge rate and a large open-issue count. Velocity is evidence of activity, not of correctness, and a loop that rewrites the instructions your agent follows is a loop that can rewrite them into something worse with a green score attached.

Do not skip the diff review. The output of a GEPA round is an evolvable text file, which means it is reviewable by a human in the way a checkpoint is not. That is the method’s best property and it is wasted if the change is merged because the score improved. Read the rewrite. A prompt that wins by adding a special case for the test set is a failure that happens to score.

The rule I took from it

The optimiser is free in the sense that matters: the loop is a hundred lines and the reflection call costs pennies. What is expensive is the thing that decides whether any of it is real, which is an evaluator with known-good answers and a held-out set you did not tune on.

Build that first. If you cannot, you do not have a self-improving agent, you have a text generator with a number next to it.

References

[1] Agrawal, Tan, Soylu, Ziems, Khare, Opsahl-Ong, Singhvi, Shandilya, Ryan, Jiang, Potts and Sen, GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning, arXiv:2507.19457: the source for Genetic-Pareto, for the reflection-over-trajectories mechanism, and for the comparison against group-relative policy optimisation and its thousands of rollouts.

[2] Reef documentation, GEPA: reflective agent harness optimization: the implementation detail this post leans on, namely parent sampling weighted by per-problem wins, one component rewritten per round, strict-better acceptance on the parent’s own batch, the score cache, and the free minibatch when the parent is the served tree.

[3] Human-Agent Society, reef, Apache-2.0: the harness evolution engine, whose recipes include the weight-training and test-time paths that ingest served requests and responses.