Agent Architectures: Control Loops, State & Planning › Planning Strategies

Planning turns high-level goals into executable sequences of actions. While simple agents might react opportunistically to each observation, advanced architectures employ explicit planning mechanisms to structure their approach to complex tasks. Planning serves several necessary functions: it reduces cognitive load by breaking complex problems into manageable pieces, it lets the agent to anticipate future needs and lay groundwork for later steps, and it gives a framework for monitoring progress and detecting when the current approach is not working.

The difference between a planning agent and a reactive agent is the difference between a surgeon who reviews the patient's file, rehearses the procedure, and assembles the necessary instruments before making the first incision, versus one who improvises at each step based only on what is immediately in front of them. For routine, well-understood tasks, the reactive approach may be adequate. For novel, complex tasks with many interdependencies, explicit planning is needed for achieving a coherent outcome.

Planning strategies fall along a spectrum from reactive to deliberative, with different approaches suited to different problem structures.

Single-step planning generates one action at a time based on the current state. The LLM examines the current observation and immediately decides the next tool to call, with no explicit plan persisting between iterations. This approach is computationally efficient and naturally responsive to environmental changes: there is no outdated plan to discard when circumstances shift, because the agent reacts fresh each time. However, single-step planning may lack global coherence for multi-step tasks. It resembles the strategy of a chess player who considers only the current board position without thinking ahead several moves. Early decisions can close off better paths that would have been available with foresight, and without a plan to refer to, the agent may lose track of the overall goal when absorbed in solving immediate subproblems.

Chain-of-thought planning explicitly generates a reasoning sequence before selecting actions. As we explored in our discussion of chain-of-thought emergence, intermediate reasoning steps improve performance on complex problems by decomposing them into manageable sub-tasks. Rather than immediately jumping to a tool call, the agent first articulates its understanding of the problem and its intended approach. This externalized reasoning serves multiple purposes: it forces the agent to be explicit about its assumptions, it creates an audit trail for debugging when things go wrong, and it often produces better results by letting the model to work through the logic step by step rather than jumping straight to a conclusion. The cost is additional tokens per iteration, which can add up over many steps.

Hierarchical task networks decompose goals into sub-goals, which further decompose into primitive actions. A research task might break down into: identify key sources, extract relevant information from each source, and synthesize findings into a coherent narrative. Each sub-goal generates its own planning context, reducing cognitive load and letting modular reasoning. This approach mirrors project management methodologies where complex deliverables are recursively divided into smaller, actionable work packages. The hierarchical structure also facilitates parallelization, as independent sub-goals can be delegated to separate worker processes running simultaneously. The challenge is correctly identifying the right decomposition: a poor decomposition can create sub-goals that are internally coherent but fail to combine into a correct overall solution.

Tree search planning explores multiple possible action sequences simultaneously, evaluating each branch's likely outcome before committing to one. This approach, common in classical AI planning and game playing, allows agents to backtrack from dead ends and select optimal paths through combinatorial search spaces. For example, when debugging code, an agent might explore several hypotheses about the bug's cause simultaneously, pursuing each line of investigation until evidence supports or refutes it, then consolidating the findings into a diagnosis. While computationally expensive, tree search protects against local optima and unexpected obstacles. Current LLM-based agents rarely implement full tree search due to cost, but prompting techniques like "let's consider several approaches" approximate it by generating multiple candidate plans before selecting one.

Plans can be represented in several formats, each with tradeoffs in expressiveness and interpretability as well as ease of execution.

Linear sequences specify an ordered list of actions such as [Search, Extract, Summarize]. This works well for straightforward workflows where each step follows naturally from the last, but struggles with conditional logic. Linear plans are easy to visualize and validate, and they give a clear sense of progress as items are checked off. However, they assume the environment will cooperate with the predetermined sequence. If the Search step returns empty results, a linear plan has no mechanism for branching to a different information source: it simply proceeds to Extract, operating on nothing.

State machines define transitions between states based on observations. The agent moves from "Researching" to "Synthesizing" only when sufficient information has been gathered, handling branching and looping naturally. State machines excel at representing workflows with clear phase transitions and validation gates. A data processing pipeline might use a state machine where records must pass quality checks before advancing to analysis stages, and failed records branch to a review queue rather than proceeding. The state machine gives explicit structure for the agent's workflow phases, which makes it easy to reason about which state the agent should be in at any point and what conditions govern transitions.

Programmatic structures stand for plans as code or pseudocode, allowing complex control flow with loops and conditionals as well as function calls. This approach treats planning as code generation, using the LLM's training on programming languages to produce structured execution plans. A programmatic plan might look like a function with conditional branches and loops expressed in natural pseudocode. This representation bridges the gap between high-level planning and executable instructions, though it requires careful sandboxing to prevent unsafe code execution when plans are translated into actual runnable code.

Static plans fail when environments change or actions produce unexpected results. A plan formulated at the outset of a task is based on assumptions about the world that may prove false as the agent gathers information. Effective agent architectures incorporate replanning triggers: conditions that invalidate current plans and initiate new planning cycles.

Common replanning triggers include tool execution failures or unexpected return values that indicate the current approach is unworkable, new information that invalidates previous assumptions about the task structure, user interventions or goal modifications that shift the objective mid-stream, and timeout conditions showing stalled progress or potential infinite loops.

Replanning frequency presents a real design tradeoff. Replanning too frequently wastes computation and creates instability, as the agent constantly second-guesses itself without making sustained progress toward the goal. But replanning too infrequently produces rigid behavior that cannot adapt to changing circumstances, like a navigation app that insists on turning left onto a road that was closed after the route was calculated. Many architectures implement opportunistic replanning: maintaining stable plans while remaining alert to significant environmental changes that warrant a strategic revision, much like a human project manager who follows a project plan but calls a team meeting when a key dependency unexpectedly fails.

The decision about when to replan can itself be part of the agent's reasoning. At each iteration, the agent can evaluate whether its current plan remains valid given the latest observation, and trigger replanning only when the gap between expected and actual outcomes exceeds a threshold. This meta-level reasoning about plans distinguishes advanced agents from brittle ones.