Planning › Goal-Directed Planning Architectures

Several architectures have been proposed for LLM-based goal-directed planning. They differ in how much reasoning is done upfront versus during execution, in how the plan is represented, and in how failures are handled. Choosing the right architecture for a given task type is one of the most consequential design decisions in building a planning agent.

The fundamental tradeoff that distinguishes these architectures is between commitment and flexibility. An architecture that commits fully to a plan before execution is efficient when the plan is correct but fragile when it is not. An architecture that reasons dynamically during execution can adapt to surprises but pays a higher per-step reasoning cost. No single architecture dominates across all task types, which is why practitioners need to understand when each one is appropriate.

A second dimension of variation is how parallelism is handled. Some architectures are inherently sequential: each step must complete before the next can begin. Others make dependencies explicit, enabling steps that do not depend on each other to run concurrently. For multi-agent systems where multiple specialized agents can act simultaneously, this distinction has a major impact on total task completion time.

The simplest architecture separates planning from execution into two distinct phases. In the planning phase, the model receives the goal and generates a complete plan: a numbered list of steps to be taken. In the execution phase, each step is executed in order by an agent with access to tools. The executing agent does not revise the plan; it simply follows it.

This separation has appealing properties. The planner can reason about the full structure of the task without being distracted by implementation details. The executor can focus on completing individual steps without worrying about the global goal. The phases can even use different models: a large, expensive model for planning and a smaller, cheaper model for execution. This model-size split is a practical optimization that some production systems use to reduce cost without sacrificing plan quality, since planning requires broad knowledge and multi-step reasoning while execution often requires only the ability to correctly invoke a tool with specified parameters.

The weakness of plan-and-execute is brittleness. Plans generated upfront cannot anticipate every contingency. When step three fails because a tool returns an unexpected error, or when the output of step two reveals that the approach planned in steps four through seven is no longer appropriate, the executor has no mechanism to adapt. The plan either fails or the system falls back to replanning from scratch, which is expensive. Plan-and-execute is therefore best suited for tasks with high predictability: the steps are known before execution begins, tool behavior is reliable, and the environment is stable.

ReWOO, introduced by Xu et al. (2023), extends the plan-and-execute idea by making dependencies between steps explicit. Instead of just listing steps, the planner writes variable names that represent the outputs of earlier steps, which later steps can reference. For example:

• Step 1: Search for the population of France. Result: #E1.
• Step 2: Search for the GDP of France. Result: #E2.
• Step 3: Calculate GDP per capita using #E1 and #E2. Result: #E3.

The planner knows from the start that step three depends on steps one and two. The executor runs steps one and two (possibly in parallel, since they are independent), collects their results, substitutes them for the variable references in step three, and executes step three with the concrete values. This makes dependencies explicit and enables safe parallelism: steps that share no variable dependencies can be run concurrently.

The name "Reasoning Without Observation" refers to the fact that the planner generates the entire plan before any observations are made. This is efficient when the plan structure is predictable, because the executor never needs to call the planner again during execution. The planner's reasoning is concentrated in a single upfront call, and the executor's job is simple substitution and dispatch. The cost is the same as plan-and-execute: the planner cannot adapt to surprising intermediate results because it committed to the plan structure before seeing any observations.

ReWOO works particularly well for structured research and question-answering tasks where the information retrieval pattern is predictable and intermediate results flow clearly from one step to the next. It works less well when the plan structure itself depends on what intermediate results look like, for example when a search might return zero results and the appropriate follow-up query depends on what was or was not found.

LLM+P (Liu et al., 2023) hybridizes language model planning with classical symbolic planning. The language model translates a natural language goal description into a formal PDDL (Planning Domain Definition Language) problem, a classical planner like FastDownward solves the formal problem and produces an optimal or near-optimal plan, and the language model translates the formal plan back into natural language actions.

PDDL is the standard language for classical AI planning. It describes a planning domain in terms of types (objects that exist in the world), predicates (properties and relationships between objects), and actions (operators that have preconditions and effects). Given a domain description and a problem specification (initial state and goal state), a PDDL planner performs state-space search to find a valid action sequence. FastDownward, one of the most widely used PDDL planners, combines multiple heuristic search strategies to find solutions efficiently.

This hybrid approach inherits the strengths of classical planning, including provable correctness, optimality guarantees, and deterministic behavior, while using the LLM to bridge the gap between natural language and formal representation. The LLM's job is translation, not planning. The planning is done by a verified solver that cannot hallucinate. LLM+P works well for well-structured domains such as logistics and scheduling, including navigation problems, where a clean formal model can be constructed. It struggles when the task involves ambiguity, open-ended information retrieval, or actions that cannot be enumerated in a formal domain description. Most real-world LLM agent tasks fall into this difficult category, which limits LLM+P's practical applicability despite its theoretical appeal.

Tree-of-Thought (ToT) planning, introduced by Yao et al. (2023), treats planning as a search problem over a tree of reasoning states. Rather than committing to a single linear plan, the model explores multiple branching possibilities simultaneously. At each decision point, the model generates several candidate next steps, evaluates each candidate (either with self-evaluation prompts or an external verifier), and expands the most promising branches.

This is significantly more expensive than linear planning because each node in the tree may require multiple model calls: one to generate candidates, one per candidate to evaluate it, and further calls to expand chosen branches. But it is also more resilient: dead ends are abandoned before they consume execution resources, and the best path through the problem space is found even when the first plausible path turns out to be wrong.

The search strategy in ToT can be breadth-first (explore all nodes at depth $d$ before advancing to depth $d+1$), depth-first (follow the most promising branch as far as possible before backtracking), or best-first (always expand the globally highest-scoring node). Each has different tradeoffs: breadth-first is more thorough but requires maintaining many candidate paths simultaneously; depth-first is efficient when good paths exist and poor paths are quickly identifiable; best-first is generally the best choice when a reliable evaluation function is available.

The evaluation function determines which branches ToT explores. If the model evaluates candidate steps poorly, ToT can waste enormous compute exploring bad branches while missing good ones. Effective evaluation prompts ask the model to reason about whether a candidate step makes progress toward the goal, whether it is logically consistent with prior steps, and whether it opens or closes future options. External verifiers, such as code execution for planning tasks that involve writing code, can provide ground-truth feedback that is more reliable than self-evaluation.

ToT is most valuable for tasks with a large search space of possible approaches where early choices significantly constrain what is possible later. Mathematical problem solving, creative writing with specific constraints, and code generation for complex algorithms are natural fits. It is overkill for straightforward retrieval and summarization tasks where the first plausible plan is usually correct.

A practical middle ground between pure plan-and-execute and fully dynamic step-by-step reasoning is to combine an initial planning step with the ReAct loop. The agent first generates a high-level plan in natural language, and then executes the plan using the ReAct thought-action-observation cycle. Each iteration of the ReAct loop is guided by the current plan step, but the agent retains the ability to revise the plan if observations suggest it is no longer valid.

This architecture is more flexible than plan-and-execute because observations can trigger plan revisions, and simpler than full tree-of-thought planning because the search tree is collapsed to a linear sequence except when explicit replanning is needed. The plan functions as a soft constraint on behavior rather than a hard specification. The agent uses it as a guide but can deviate when the situation requires it.

PlanReAct is the most commonly used architecture in practical deployments as of 2024. Most production agent systems, including various implementations of LangGraph, AutoGPT-style systems, and the OpenAI Assistants API, use some version of this pattern. The combination of upfront deliberation with ongoing adaptation captures most of the benefit of pure planning while retaining most of the flexibility of pure reaction. Its weakness is that plan revisions can be expensive, since each revision requires a full model call with substantial context, and in adversarial or rapidly changing environments the agent may spend more time replanning than executing.

An extension of explicit dependency management that is gaining traction in production systems is representing plans as directed acyclic graphs (DAGs) rather than lists. In a list plan, steps are ordered linearly and parallelism must be inferred from the absence of dependencies. In a graph plan, nodes are tasks and directed edges represent data dependencies: an edge from node A to node B means B requires A's output as input.

Graph-based plans make the parallel structure immediately visible and machine-readable without requiring any inference. An orchestration engine reading the graph can trivially identify which nodes have all their dependencies satisfied and dispatch them simultaneously. When a node fails, the graph makes it clear exactly which downstream nodes are blocked, enabling targeted recovery reasoning rather than full replanning.

The LangGraph library implements this pattern directly. Workflows are defined as graphs where nodes are agent steps (model calls or tool invocations) and edges encode control flow and data dependencies. This representation also supports cycles, which pure DAG plans cannot express, enabling loops with exit conditions that are essential for tasks requiring iterative refinement.