Planning › Plan Execution and Monitoring

Generating a plan is only half the problem. Executing it correctly requires managing the flow of control, monitoring progress, handling intermediate results, and detecting when something has gone wrong. These engineering concerns are as important as the planning algorithms themselves, and in practice they often determine whether a planning agent works reliably in production.

As each step of the plan is executed, its result must be stored and made available to subsequent steps. The planning agent needs to maintain a context object that records:

• The original goal
• The full plan with step statuses (pending, in progress, completed, failed)
• The outputs of each completed step
• Any observations or errors encountered during execution

This context object is the agent's working memory. It prevents the agent from forgetting what it was doing and ensures that later steps have access to all information generated by earlier steps. In practice, this context is often passed as part of the system prompt or injected into the model's context window on each step. For very long plans where the accumulated context would exceed the context window, selective summarization of completed steps keeps the total size manageable while preserving the information most relevant to current execution.

A significant engineering challenge is context window management. As plans grow longer and step outputs accumulate, the total number of tokens in context can exceed the model's limit. Strategies for managing this include summarizing the outputs of completed steps rather than preserving them verbatim, keeping only the most relevant recent context, and using an external memory store (as discussed in the Agent Memory chapter) to offload history that does not need to be in the immediate context. A common heuristic is to keep the last $k$ steps in full and maintain compressed summaries of everything older, where $k$ is chosen to be large enough to handle the typical dependency depth of the plan.

The choice of what to include in context at each step is not trivial. Including too little risks the executor making decisions without relevant information. Including too much fills the context window with irrelevant content that dilutes the signal the model can act on. The optimal context for a given step is the goal, the current step description, the outputs of direct predecessor steps, and any recent observations that might affect the current step. Everything else can be deferred to external memory.

A well-designed planning agent monitors its own progress. After each step completes, the agent checks whether the step's output is consistent with what was expected, whether the plan's preconditions for the next step have been satisfied, and whether any new information suggests the plan should be revised.

This monitoring can be implemented as a simple check after each execution: the agent is prompted to assess whether the step output is valid and whether the plan should proceed, be modified, or be abandoned. More sophisticated implementations use a separate critic model to evaluate outputs, or define explicit success criteria for each step that can be checked programmatically. Programmatic success criteria, such as verifying that a search step returned at least one result or that a code execution step produced no errors, are preferable when they can be defined because they do not require an additional model call and cannot be fooled by plausible-sounding but incorrect model self-assessment.

Progress monitoring also is a form of early stopping. If a plan's first few steps produce unexpected results that make the goal unachievable or nonsensical, catching this early saves the cost of executing the remaining steps. An agent researching a factual question that discovers in step two that its initial premise was wrong should stop and reflect before replanning rather than continuing to execute a plan built on an incorrect assumption.

Tool failures are one of the most common sources of plan breakdown. APIs become unavailable, search queries return empty results, code execution raises exceptions, rate limits trigger. The plan executor must distinguish between three fundamentally different types of failure, because the correct response differs for each:

• Recoverable failures: A tool call fails due to a transient error (network timeout, rate limit). The right response is to retry with backoff. These failures are not about the plan being wrong; they are about infrastructure instability. A well-designed executor handles them without involving the planner at all.
• Fixable failures: The tool call fails because the input was malformed or the query was poorly specified. The right response is to revise the tool call. The plan structure is correct, but the specific parameters need adjustment. This is a local repair: the planner needs to modify the step's tool input but can leave the rest of the plan intact.
• Plan-invalidating failures: The tool call reveals that a fundamental assumption in the plan is wrong. A search returns evidence that the factual premise of the plan does not hold, or a required API endpoint does not exist. The right response is replanning because the plan's structure is incorrect; changing a single step's parameters cannot fix it.

Distinguishing these failure types requires the agent to reason about why a failure occurred, not just that it occurred. This is a place where the language model's reasoning capability is especially valuable: given an error message and the context in which it occurred, the model can usually identify which category the failure falls into and propose an appropriate response. A 429 Too Many Requests error is clearly recoverable. A 404 Not Found error on an API endpoint the plan required is plan-invalidating. A malformed query parameter that produces no results is fixable.

Retry policies for recoverable failures should implement exponential backoff to avoid overwhelming already-stressed infrastructure. A simple policy retries after 1 second, then 2 seconds, then 4 seconds, then declares the step permanently failed. The maximum retry count is a hyperparameter that trades resilience for latency: higher values handle longer transient outages but increase wait time in the common case.

Beyond hard tool failures, planning agents must also detect semantic failures: cases where a tool call technically succeeds but returns content that does not satisfy the step's requirements. A search that returns results about a different topic than intended is a semantic failure. A code generation step that produces syntactically valid code that does not implement the specified algorithm is a semantic failure.

Semantic failure detection requires the agent to verify that step outputs satisfy their intended purpose, not just that the tool call returned without an error. This is harder than detecting hard failures because it requires understanding what the step was supposed to produce, which requires maintaining a clear specification of each step's expected output in the plan. When steps are specified only as vague descriptions like "gather information about X", semantic failure detection is nearly impossible because there is no clear criterion for what would count as success. Well-specified steps with explicit output requirements make semantic failure detection tractable.