Capability Frontiers › Planning and Agency

The shift from language model to agent represents one of the most significant capability frontiers. An agent does not just answer questions: it pursues goals, takes actions, observes outcomes, and adjusts behavior accordingly. This requires capabilities that language models were not originally designed to exhibit.

Effective planning begins with decomposition: breaking a complex goal into manageable subgoals. When asked to "write a research report on climate change," an intelligent system should recognize that this requires gathering information, organizing themes, writing sections, and reviewing for coherence. These are distinct subtasks that can be executed sequentially, and each subtask is substantially simpler than the original task.

Modern language models have shown surprising capability for task decomposition when prompted appropriately. Given a complex goal, they can generate reasonable plans with appropriate granularity. They naturally produce hierarchical plans, identifying major phases before specifying substeps within each phase, which aligns with how humans approach complex projects.

The challenge comes in execution: pursuing the plan, handling unexpected obstacles, and adjusting when subgoals fail. A plan generated at the beginning of a task may become invalid partway through due to information discovered during execution. A model that can generate a good initial plan but cannot revise it intelligently when circumstances change is less useful than one that can adapt. This requires maintaining a representation of the current state of the task, tracking which subgoals have been completed and which remain, and identifying when the original plan needs revision.

Research on planning with language models has identified several failure modes worth understanding. Models sometimes generate plans that look locally reasonable but are globally inconsistent: one step requires resources that another step has already consumed, or two steps that cannot be executed simultaneously are not flagged as conflicting. They struggle with resource constraints when those constraints interact in non-obvious ways. They can be overly optimistic, generating plans that assume subgoals will succeed without accounting for failure cases or providing fallback strategies. And they sometimes produce plans with the wrong granularity, either too coarse to be actionable or too fine-grained to be strategically coherent.

A fundamental limitation for language model agents is working memory. The context window imposes a hard limit on how much information the model can hold in view at once. For short tasks, this is rarely a problem. For long-horizon tasks, where progress must be tracked over hundreds of steps, it becomes a significant constraint.

The context window functions as external working memory for the model: everything the model can consider simultaneously must fit within it. This is qualitatively different from human working memory, which is supplemented by long-term memory that can be recalled based on cues. For a human working on a long project, relevant context from earlier in the project is retrievable when needed; for a language model agent, earlier context is simply gone once it scrolls out of the context window unless explicitly saved and retrieved.

Various approaches have addressed this limitation. Retrieval-augmented generation allows models to query an external knowledge base rather than relying solely on the context window, effectively giving the model a form of long-term memory. Memory augmentation systems store summaries of past actions in external buffers that can be retrieved when relevant, preserving task-relevant context across context window boundaries. Hierarchical planning approaches use one model to generate high-level plans and another to execute individual steps, reducing the cognitive load on any single forward pass and allowing long-horizon tasks to be decomposed into shorter-horizon segments.

None of these approaches fully solves the long-horizon planning problem, but together they extend what agents can reliably accomplish. The current state of the art can handle tasks that take tens of steps to complete with reasonable reliability. Tasks requiring hundreds of interdependent steps remain challenging because context length is limited and errors accumulate over long horizons. A small probability of error at each step, multiplied over many steps, can quickly dominate overall reliability.

The question of how to build agents that remain reliable over long horizons is one of the active research areas in AI systems. Promising directions include planning algorithms that better handle failure that explicitly model uncertainty and plan for failure cases, better integration of symbolic planning tools that can enforce global consistency, and improved self-monitoring mechanisms that detect when the agent has drifted from its intended plan.

A capability closely related to planning is self-correction: the ability to evaluate one's own outputs, identify errors, and produce improved versions. This matters because language models make mistakes, and a model that can catch and fix its mistakes is more reliable than one that cannot. The ideal agent would be one that produces an initial output, checks it, finds errors, revises them, and iterates until the output meets quality criteria.

Self-correction turns out to be harder than it sounds. Studies have found that when models are simply asked to check their own answers, they often validate incorrect responses rather than catching errors. The same reasoning failure that produced the error often persists during review. This makes intuitive sense: if the model failed to reason correctly the first time due to some systematic bias or missing knowledge, reviewing the answer does not fix that underlying issue.

Effective self-correction seems to require some form of separation between generation and evaluation. Approaches that work better include providing explicit feedback from a trained critic model that evaluates outputs on specific criteria, rather than asking the generating model to evaluate its own outputs. Breaking review into smaller targeted checks rather than open-ended review also helps: "check whether the solution handles the edge case where the input is empty" is a more tractable review task than "check whether the solution is correct."

The most reliable form of self-correction involves grounding: checking outputs against external verifiable information. A model that generates code and then executes it gets concrete feedback about whether the code works, feedback that is unambiguous and not subject to the same reasoning errors that affected generation. A model that generates a mathematical claim and then checks it against a symbolic solver gets reliable feedback about correctness. This kind of grounded self-correction is substantially more reliable than pure language-based review and represents one of the strongest arguments for tool-augmented systems over pure language model agents.

Reflection, a broader capability, involves correcting specific outputs and learning from patterns of errors over time. An agent that notices it consistently fails on problems involving negative numbers might hypothesize a systematic weakness and actively seek to avoid such problems or compensate for them. This kind of meta-learning in context is an active research area, with approaches including explicit reflection prompts that ask the model to identify what went wrong and how to avoid it in the future.