Evolving Agents

Evolving Agents

How agents turn experience into procedures, and procedures into predictions.

Leonardo Ranaldi, University of Edinburgh and University of Rome Tor Vergata. A technical post on EVA (ACL 2026) and WEVA (EMNLP 2026).

Figure 1. EVA and WEVA share one representation. EVA turns a trajectory into a quasi-symbolic abstraction, stores it after a verified success or an instructive failure, and retrieves it on later tasks. WEVA uses the same state to predict what a candidate action would cause, and prunes an action whose predicted consequence breaks a precondition before it reaches the environment.

Language agents increasingly solve streams of related tasks in the same environment. A shopping agent searches the same catalogue, an assistant edits the same set of files, a travel agent books through the same sites. Each task leaves experience that could make the next one easier. In practice, agents keep little of it, and they repeat similar mistakes across tasks that share the same structure.

A common approach is to store experience as text, in the form of trajectories, summaries or skill documents. Stored text, however, mixes the procedure with the details of the episode in which it was learned. When the next task looks different on the surface, it is hard to tell which part still applies, and hard to check whether it was followed.

In this post we describe two frameworks that take a different route. Both keep experience in a quasi-symbolic form, where a state is organised around entities, transitions, constraints and outcomes, and the content of each field is written in natural language. EVA (Evolving Agents) uses this state to decide what to remember and how to reuse it. WEVA (World-Model Evolving Agents) uses the same state to predict what an action would cause.

This sounds straightforward, but there is a catch. Remembering what happened helps only after the fact, since the agent learns from an error once the error has occurred, and some errors cannot be undone. Our main finding is that the representation that makes experience reusable is also what makes it predictive and checkable. A structured state lets EVA retrieve a procedure and adapt it to a new task. It also lets WEVA compare a predicted state with the observed one, field by field, and reject a consequence before experiencing it.

At a glance

  • Experience needs a form. A typed state of entities, transitions, constraints and outcomes can be compared, retrieved and reused across tasks.
  • Memory keeps successes and failures. Successes supply reusable transitions, and localised failures supply constraints on what to avoid.
  • Prediction comes before action. A learned world model rolls candidate actions forward, and predicted violations are pruned before execution.
  • Trust regulates prediction. The agreement between predicted and observed states tells the agent when its expectations have become unreliable.

Problem statement

We consider an agent that solves tasks in an interactive environment, for example a web shop, a household simulator or a set of connected applications. At each step the agent observes the environment, takes an action and receives feedback. We want two things from such an agent. First, experience gathered on earlier tasks should make later tasks easier. Second, the agent should avoid actions whose consequences cannot be undone, such as overwriting a file or replacing a booking.

Both goals depend on how experience is represented. Natural-language traces are expressive, but they provide no fixed place where a stored claim can be compared with what happens next. Fully symbolic representations make that comparison explicit, at the cost of defining everything in advance. Quasi-symbolic abstractions settle on an intermediate form . The roles are fixed and the content is induced from interaction.

Quasi-symbolic abstraction, A = (V, T, C, E)

Booking a flight when the return is already booked

V, entities
Outbound flight options, return booking, payment page
T, transitions
Search flights, select one, pay
C, constraints
The return booking stays unchanged
E, outcomes
Outbound not booked yet. Progress 0 of 2.
Figure 2. One state, four roles. The four fields keep the same role in every environment. Two states can therefore be compared field by field, and a proposed action can be checked against the constraints.

EVA: turning experience into procedures

EVA organises the agent around three modules, the Perceptor, the Controller and the Actor. The three modules read and write the same quasi-symbolic abstraction, which also serves as the interface to memory . The first tab of Figure 1 follows one complete cycle.

The Perceptor reads the trajectory

Given the goal, the current observation and a recent window of actions and feedback, the Perceptor writes the active abstraction. It also reports how confident it is in what it has written, a signal that the Controller reads at the next step.

The Controller decides how to proceed

The Controller does not choose environment actions. It reads a compact summary of the abstraction, namely the confidence, the share of contradicted constraints, the novelty against memory and the progress made, and it selects one of four meta-actions.

Act

proceed with the Actor under the current abstraction

Abstract

rebuild the abstraction over a longer trajectory

Retrieve

look for compatible experience in memory

Rollback

restore the last consistent abstraction

A rollback restores the reasoning state. It does not undo actions already executed in the environment, and the Actor replans from the current observation.

What enters memory, and what leaves it

An abstraction is stored after a verified success or an instructive failure, together with a link to its trajectory and an outcome label. When a stored abstraction is retrieved, its roles are bound to the entities of the current task. Successful abstractions contribute reusable transitions. Failure-derived abstractions contribute constraints, and only when the failure can be traced to a specific action through environment feedback, a verifier or a rollback.

Memory is also selective over time. Each entry keeps a record of how often its reuse was followed by a success, discounted by how long ago that happened. Entries that have been reused often enough without helping are pruned.

Two timescales of learning

The Controller is calibrated before interaction and then held fixed, whilst the Perceptor and the Actor keep learning from the trajectories the agent produces. Keeping the Controller fixed avoids the instability of updating meta-control and behaviour at the same time. A frozen configuration, EVA-Doc, keeps all weights fixed and adapts only through the abstractions it builds, retrieves and consolidates.

WEVA: from remembering to predicting

EVA improves which action the agent proposes. The consequence of that action, however, is observed only after execution. Over long horizons this becomes critical, since an error may surface only when the state in which it could have been prevented is no longer reachable. WEVA adds a world model, and the agent evaluates candidate actions against it before acting . The second tab of Figure 1 follows the procedure on the flight request.

A state that can be predicted

WEVA keeps the entities, preconditions and outcomes of the abstraction as its predictive state. A transition model learns to map a state and a candidate action to the state expected to follow. No annotation is required, since after every executed action the next observation shows what actually happened. The Perceptor is also rewarded when its states are easier to predict, which encourages it to keep the information that governs how the environment changes.

Verified lookahead

Before acting, the Actor proposes a few candidate actions and the world model rolls them forward, up to two steps in our experiments. Each predicted transition is checked against the active preconditions. A branch is discarded as soon as one of its predicted steps breaks a precondition, even when its first step looks harmless. Among the surviving branches, WEVA executes only the first action of the branch with the highest predicted progress. When no branch survives, the Actor falls back to its policy without lookahead.

Model trust

Lookahead is useful only as long as the world model still describes the environment. WEVA measures the agreement between predicted and observed states over a window of executed steps, and we call this signal model trust. High trust supports lookahead. As trust declines, direct interaction becomes more informative than prediction, and under a sustained decline the Controller rebuilds the state over a broader trajectory.

Correcting the world model

Prediction error is useful when it changes later decisions. A mismatch that recurs under the same action and in structurally similar conditions becomes a candidate precondition. The candidate is admitted only when it rejects at least one observed failure without contradicting an earlier successful transition, which keeps isolated accidents out of the world model.

What the experiments show

We evaluate both frameworks on interactive planning environments (ALFWorld, ScienceWorld, WebShop and AppWorld) and on a variant of ScienceWorld in which the environment changes without notice during interaction. The full tables are in the two papers. Here we keep the three observations that matter for the argument of this post.

The form of memory matters

The gains of EVA depend on the form in which experience is kept. Removing the quasi-symbolic abstraction has a larger effect than removing the Controller or the memory, and keeping raw trajectories is close to having no abstraction at all. The typed state is the common structure through which perception, memory and control interact .

Figure 3. The representation matters. ALFWorld success when one component is removed or the memory format is replaced.

Failures are worth remembering. A configuration that stores only verified successes stays close to the full model on static benchmarks and falls behind when the environment changes. Constraints derived from localised failures are what the agent reuses to recover.

Predicting before acting helps

WEVA improves over EVA across the environments, and the benefit is most apparent when the dynamics change or when tasks chain several preconditions across applications. Search over predicted states raises task success, while checking predicted transitions against active preconditions removes most of the verified violations .

Figure 4. WEVA against EVA. Task outcome across four environments, with the same backbone and interaction budget.

The agent can notice that the world has changed

An agent can keep completing tasks for a while after its expectations have become stale. In the shifted environment, model trust falls below its alert level before task success degrades. As trust decreases, the Controller moves from lookahead towards direct interaction and towards rebuilding the state, and a predicted side effect on a file or a booking that should stay untouched can be rejected before it happens.

Closing thoughts and future work

In this post we described two frameworks built on the same idea. EVA distils experience into quasi-symbolic abstractions and uses them to remember, retrieve and regulate. WEVA turns the same abstractions into a predictive model and checks consequences before acting. A central takeaway is that the way experience is represented determines what an agent can later do with it.

Several limits remain. Building abstractions and running lookahead add computation at every step. The schema of the abstraction is designed by hand. Confidence, model trust and acquired preconditions are approximate signals obtained from interaction, and they are not calibrated estimates.

These limits lead to open questions. Can the Controller keep learning during interaction instead of being fixed after calibration? Can the structure of the state be induced instead of designed? How should model trust be calibrated? In multi-agent settings, can the disagreement between what an agent reports and what another agent predicts help decide whether shared information is reliable? The essay Adaptive Reasoning Agency places these questions in a broader research programme.

Case studies

A. What EVA keeps from a WebShop task

The stored abstraction keeps the procedure and leaves out the product. The task below ends in a successful purchase. What enters memory is a search, verify and purchase procedure that a later request can reuse by replacing the category and the candidate product.

Goal
Find a product matching the query “grocery cookies”.
Selected product
Walkers Shortbread Saltire Shortbread Rounds Tin, 120 g, $18.95.
Category
Grocery and Gourmet Food, Breads and Bakery, Cookies, Shortbread.
Options
None. The item requires no size, flavour or colour selection.

B. WEVA on a WorldBench task

The error is the same in both runs, and only one of them executes it. The folder contains two ledgers that look alike. An agent that acts first edits the old copy and discovers the error after the file has changed. WEVA predicts the outcome of each candidate and prunes that edit.

Instruction
“Record the short delivery in the stock ledger and leave a note for the pharmacist.”
Files
delivery_note (60 Amoxicillin ordered, 40 delivered), ledger (the current ledger, the target), ledger_2025 (last year’s copy, which has to stay as it is).
Success requires
Row 4 of the ledger set to short delivery, a note naming Amoxicillin, and no other file changed.

Papers and citation

    If this post is useful for your work, please consider citing the two papers.

    @inproceedings{ranaldi-2026-evolving,
      title     = {Evolving Agents},
      author    = {Ranaldi, Leonardo},
      booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
      year      = {2026},
      pages     = {16561--16569},
      publisher = {Association for Computational Linguistics}
    }
    
    @inproceedings{ranaldi-2026-world,
      title     = {World-Model Evolving Agents},
      author    = {Ranaldi, Leonardo},
      booktitle = {Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing},
      year      = {2026},
      pages     = {49279--49294},
      publisher = {Association for Computational Linguistics}
    }