Evolving Agents
How agents turn experience into procedures, and procedures into predictions.
Leonardo Ranaldi, University of Edinburgh and University of Rome Tor Vergata. A technical post on EVA (ACL 2026) and WEVA (EMNLP 2026).
Language agents increasingly solve streams of related tasks in the same environment. A shopping agent searches the same catalogue, an assistant edits the same set of files, a travel agent books through the same sites. Each task leaves experience that could make the next one easier. In practice, agents keep little of it, and they repeat similar mistakes across tasks that share the same structure.
A common approach is to store experience as text, in the form of trajectories, summaries or skill documents. Stored text, however, mixes the procedure with the details of the episode in which it was learned. When the next task looks different on the surface, it is hard to tell which part still applies, and hard to check whether it was followed.
In this post we describe two frameworks that take a different route. Both keep experience in a quasi-symbolic form, where a state is organised around entities, transitions, constraints and outcomes, and the content of each field is written in natural language. EVA (Evolving Agents) uses this state to decide what to remember and how to reuse it. WEVA (World-Model Evolving Agents) uses the same state to predict what an action would cause.
This sounds straightforward, but there is a catch. Remembering what happened helps only after the fact, since the agent learns from an error once the error has occurred, and some errors cannot be undone. Our main finding is that the representation that makes experience reusable is also what makes it predictive and checkable. A structured state lets EVA retrieve a procedure and adapt it to a new task. It also lets WEVA compare a predicted state with the observed one, field by field, and reject a consequence before experiencing it.
At a glance
- Experience needs a form. A typed state of entities, transitions, constraints and outcomes can be compared, retrieved and reused across tasks.
- Memory keeps successes and failures. Successes supply reusable transitions, and localised failures supply constraints on what to avoid.
- Prediction comes before action. A learned world model rolls candidate actions forward, and predicted violations are pruned before execution.
- Trust regulates prediction. The agreement between predicted and observed states tells the agent when its expectations have become unreliable.
Problem statement
We consider an agent that solves tasks in an interactive environment, for example a web shop, a household simulator or a set of connected applications. At each step the agent observes the environment, takes an action and receives feedback. We want two things from such an agent. First, experience gathered on earlier tasks should make later tasks easier. Second, the agent should avoid actions whose consequences cannot be undone, such as overwriting a file or replacing a booking.
Both goals depend on how experience is represented. Natural-language traces are expressive, but they provide no fixed place where a stored claim can be compared with what happens next. Fully symbolic representations make that comparison explicit, at the cost of defining everything in advance. Quasi-symbolic abstractions settle on an intermediate form . The roles are fixed and the content is induced from interaction.
Quasi-symbolic abstraction, A = (V, T, C, E)
Booking a flight when the return is already booked
- V, entities
- Outbound flight options, return booking, payment page
- T, transitions
- Search flights, select one, pay
- C, constraints
- The return booking stays unchanged
- E, outcomes
- Outbound not booked yet. Progress 0 of 2.
EVA: turning experience into procedures
EVA organises the agent around three modules, the Perceptor, the Controller and the Actor. The three modules read and write the same quasi-symbolic abstraction, which also serves as the interface to memory . The first tab of Figure 1 follows one complete cycle.
The Perceptor reads the trajectory
Given the goal, the current observation and a recent window of actions and feedback, the Perceptor writes the active abstraction. It also reports how confident it is in what it has written, a signal that the Controller reads at the next step.
The Controller decides how to proceed
The Controller does not choose environment actions. It reads a compact summary of the abstraction, namely the confidence, the share of contradicted constraints, the novelty against memory and the progress made, and it selects one of four meta-actions.
Act
proceed with the Actor under the current abstraction
Abstract
rebuild the abstraction over a longer trajectory
Retrieve
look for compatible experience in memory
Rollback
restore the last consistent abstraction
A rollback restores the reasoning state. It does not undo actions already executed in the environment, and the Actor replans from the current observation.
What enters memory, and what leaves it
An abstraction is stored after a verified success or an instructive failure, together with a link to its trajectory and an outcome label. When a stored abstraction is retrieved, its roles are bound to the entities of the current task. Successful abstractions contribute reusable transitions. Failure-derived abstractions contribute constraints, and only when the failure can be traced to a specific action through environment feedback, a verifier or a rollback.
Memory is also selective over time. Each entry keeps a record of how often its reuse was followed by a success, discounted by how long ago that happened. Entries that have been reused often enough without helping are pruned.
Two timescales of learning
The Controller is calibrated before interaction and then held fixed, whilst the Perceptor and the Actor keep learning from the trajectories the agent produces. Keeping the Controller fixed avoids the instability of updating meta-control and behaviour at the same time. A frozen configuration, EVA-Doc, keeps all weights fixed and adapts only through the abstractions it builds, retrieves and consolidates.
WEVA: from remembering to predicting
EVA improves which action the agent proposes. The consequence of that action, however, is observed only after execution. Over long horizons this becomes critical, since an error may surface only when the state in which it could have been prevented is no longer reachable. WEVA adds a world model, and the agent evaluates candidate actions against it before acting . The second tab of Figure 1 follows the procedure on the flight request.
A state that can be predicted
WEVA keeps the entities, preconditions and outcomes of the abstraction as its predictive state. A transition model learns to map a state and a candidate action to the state expected to follow. No annotation is required, since after every executed action the next observation shows what actually happened. The Perceptor is also rewarded when its states are easier to predict, which encourages it to keep the information that governs how the environment changes.
Verified lookahead
Before acting, the Actor proposes a few candidate actions and the world model rolls them forward, up to two steps in our experiments. Each predicted transition is checked against the active preconditions. A branch is discarded as soon as one of its predicted steps breaks a precondition, even when its first step looks harmless. Among the surviving branches, WEVA executes only the first action of the branch with the highest predicted progress. When no branch survives, the Actor falls back to its policy without lookahead.
Model trust
Lookahead is useful only as long as the world model still describes the environment. WEVA measures the agreement between predicted and observed states over a window of executed steps, and we call this signal model trust. High trust supports lookahead. As trust declines, direct interaction becomes more informative than prediction, and under a sustained decline the Controller rebuilds the state over a broader trajectory.
Correcting the world model
Prediction error is useful when it changes later decisions. A mismatch that recurs under the same action and in structurally similar conditions becomes a candidate precondition. The candidate is admitted only when it rejects at least one observed failure without contradicting an earlier successful transition, which keeps isolated accidents out of the world model.
What the experiments show
We evaluate both frameworks on interactive planning environments (ALFWorld, ScienceWorld, WebShop and AppWorld) and on a variant of ScienceWorld in which the environment changes without notice during interaction. The full tables are in the two papers. Here we keep the three observations that matter for the argument of this post.
The form of memory matters
The gains of EVA depend on the form in which experience is kept. Removing the quasi-symbolic abstraction has a larger effect than removing the Controller or the memory, and keeping raw trajectories is close to having no abstraction at all. The typed state is the common structure through which perception, memory and control interact .
Failures are worth remembering. A configuration that stores only verified successes stays close to the full model on static benchmarks and falls behind when the environment changes. Constraints derived from localised failures are what the agent reuses to recover.
Predicting before acting helps
WEVA improves over EVA across the environments, and the benefit is most apparent when the dynamics change or when tasks chain several preconditions across applications. Search over predicted states raises task success, while checking predicted transitions against active preconditions removes most of the verified violations .
The agent can notice that the world has changed
An agent can keep completing tasks for a while after its expectations have become stale. In the shifted environment, model trust falls below its alert level before task success degrades. As trust decreases, the Controller moves from lookahead towards direct interaction and towards rebuilding the state, and a predicted side effect on a file or a booking that should stay untouched can be rejected before it happens.
Closing thoughts and future work
In this post we described two frameworks built on the same idea. EVA distils experience into quasi-symbolic abstractions and uses them to remember, retrieve and regulate. WEVA turns the same abstractions into a predictive model and checks consequences before acting. A central takeaway is that the way experience is represented determines what an agent can later do with it.
Several limits remain. Building abstractions and running lookahead add computation at every step. The schema of the abstraction is designed by hand. Confidence, model trust and acquired preconditions are approximate signals obtained from interaction, and they are not calibrated estimates.
These limits lead to open questions. Can the Controller keep learning during interaction instead of being fixed after calibration? Can the structure of the state be induced instead of designed? How should model trust be calibrated? In multi-agent settings, can the disagreement between what an agent reports and what another agent predicts help decide whether shared information is reliable? The essay Adaptive Reasoning Agency places these questions in a broader research programme.
Case studies
A. What EVA keeps from a WebShop task
The stored abstraction keeps the procedure and leaves out the product. The task below ends in a successful purchase. What enters memory is a search, verify and purchase procedure that a later request can reuse by replacing the category and the candidate product.
- Goal
- Find a product matching the query “grocery cookies”.
- Selected product
- Walkers Shortbread Saltire Shortbread Rounds Tin, 120 g, $18.95.
- Category
- Grocery and Gourmet Food, Breads and Bakery, Cookies, Shortbread.
- Options
- None. The item requires no size, flavour or colour selection.
- Perceptor
- Identifies the central requirement as a category match. The target has to be a grocery item classified as cookies. No price limit or option is specified.
- Controller
- Lets the Actor search for the category. Once a candidate is retrieved, it requires the product page to be inspected before purchase.
- Actor
- search[grocery cookies], inspect the candidates, click[Walkers Shortbread …], verify category, price and options, click[Buy Now].
- Resolution
- The item satisfies the goal, and no option or conflicting attribute remains unresolved before purchase.
- Type
- Product-selection procedure, stored with a SUCCESS label.
- Goal pattern
- Find and purchase a product that matches a requested category.
- Procedure (T)
- Form a query from the category, inspect the candidates, open the most relevant page, verify title, category, price and options, purchase once every requirement has been checked.
- Constraints (C)
- The category is verified on the product page. Any required option is selected before purchase. No purchase while an explicit requirement remains unverified.
- New request
- “Find grocery crackers.”
- Retrieve and bind
- EVA retrieves the abstraction, replaces the target category and the candidate product, and keeps the search, inspect, verify and purchase procedure.
- What changes
- Only the values of the entities. The transitions and the constraints are reused as they are.
B. WEVA on a WorldBench task
The error is the same in both runs, and only one of them executes it. The folder contains two ledgers that look alike. An agent that acts first edits the old copy and discovers the error after the file has changed. WEVA predicts the outcome of each candidate and prunes that edit.
- Instruction
- “Record the short delivery in the stock ledger and leave a note for the pharmacist.”
- Files
- delivery_note (60 Amoxicillin ordered, 40 delivered), ledger (the current ledger, the target), ledger_2025 (last year’s copy, which has to stay as it is).
- Success requires
- Row 4 of the ledger set to short delivery, a note naming Amoxicillin, and no other file changed.
| Step | Action | What happens |
|---|---|---|
| 1 | Read the delivery note | 40 of 60 delivered. |
| 2 | Edit row 4 of ledger_2025 | The wrong file is modified. |
| 3 | Write the note | The task checks pass. |
| Evaluation | Constrained task success 0 The old copy has changed. |
| Candidate action | Predicted state (not executed) | Check |
|---|---|---|
| Edit ledger_2025 | The old copy has changed. | Breaks C, pruned |
| Finish the task | Task closed, progress 0 of 2. | No progress |
| Edit ledger | Row 4 set to short delivery, progress 1 of 2. | Consistent, executed |
Only the executed action reaches the environment. The observed state matches the prediction, and model trust stays high. After the note is written, the ledger is updated, the old copy is untouched and constrained task success is 1.
Papers and citation
If this post is useful for your work, please consider citing the two papers.
@inproceedings{ranaldi-2026-evolving,
title = {Evolving Agents},
author = {Ranaldi, Leonardo},
booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
year = {2026},
pages = {16561--16569},
publisher = {Association for Computational Linguistics}
}
@inproceedings{ranaldi-2026-world,
title = {World-Model Evolving Agents},
author = {Ranaldi, Leonardo},
booktitle = {Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing},
year = {2026},
pages = {49279--49294},
publisher = {Association for Computational Linguistics}
}