Agent Architectures: Control Loops, State & Planning › Limitations and Impact

Agent architectures, while powerful, introduce significant challenges that practitioners must manage carefully. The flexibility that lets complex problem-solving also creates vectors for failure modes that simpler systems avoid entirely. Understanding these limitations is needed for deploying agents responsibly and effectively in production environments.

Perhaps the most pressing practical concern is that errors compound across iterations. In a single-step function-calling system, an error is localized and easy to attribute: either the tool call succeeded or it failed, and the failure is immediately visible. In a multi-step agent loop, an incorrect inference in step 3 might not manifest obviously until step 7, by which time the agent has built elaborate reasoning on false premises and taken several actions based on that reasoning. Without explicit verification steps or self-consistency checks, agents can confidently pursue entirely wrong trajectories while appearing to make progress.

This problem is compounded by the LLM's tendency toward confident generation even when uncertain. A language model that is unsure whether a web search returned reliable results may still generate a confident synthesis, masking the uncertainty from the planning logic. Building in explicit uncertainty tracking, where the agent maintains confidence scores for key claims and triggers human review when confidence drops below a threshold, is one mitigation strategy, but it adds complexity and is itself susceptible to miscalibration.

The figure illustrates why validation checkpoints matter even when they add cost. At a modest 5% per-step error rate, unvalidated agents face a 64% chance of accumulated error by iteration 20. Periodic validation resets this accumulation, keeping cumulative error probability manageable across long-running tasks. The tradeoff is the additional tokens and latency of the validation steps themselves.

Agent architectures face a basic optimization problem in deciding how to allocate their iteration budget between exploration and exploitation. Too much exploration, trying different approaches, searching for additional information, considering alternative plans, wastes tokens and time without converging on a solution. Too much exploitation, committing early to the first promising approach, leads to local optima where agents miss better solutions because they failed to consider alternatives.

Current architectures rely heavily on heuristics to balance this tradeoff: fixed iteration limits, simple termination conditions based on the absence of new information, and greedy action selection that always picks the highest-confidence next step. These heuristics work reasonably well for well-structured tasks but can fail badly on tasks with deceptive local optima, where the most obvious approach turns out to be a dead end that only becomes apparent after significant investment.

More advanced approaches might employ tree search, Monte Carlo rollouts, or learned value functions to guide exploration, similar to the techniques used in reinforcement learning. However, these methods increase computational cost substantially and require either a simulator of the environment (for tree search) or extensive training data (for learned value functions). The practical reality is that most deployed agents use greedy, heuristic-based planning because it is the only approach that remains tractable at inference time with current hardware.

Our implementation demonstrated simple working memory, but real-world agents managing database transactions, file systems, and API states face consistency challenges that echo those in distributed systems. When an agent executes a sequence of operations, reading a file, processing it, and writing results, what happens if the write fails? Should the agent roll back previous operations? Should it retry? Should it escalate to human operators? These are the distributed systems problems that database engineers have addressed with ACID transactions and two-phase commit protocols, but those solutions assume a controlled database environment. Agent architectures operate across heterogeneous external systems that may not give transactional guarantees.

Building reliable state management requires importing lessons from distributed computing: idempotency keys to prevent duplicate operations when retrying failed steps, circuit breakers to fail fast when dependent services are unavailable, and saga patterns to manage distributed transactions that span multiple systems. Most current agent frameworks address these challenges incompletely, placing the burden of state consistency on the application developer rather than giving reliable infrastructure support.

Each loop iteration requires at least one LLM inference call. An agent that iterates 10 times to solve a problem costs 10 times more in API fees and generates 10 times the latency compared to a single-shot solution. This creates real pressure to minimize iterations, sometimes at the expense of solution quality. For consumer applications with sub-second response expectations, agents requiring multiple reasoning steps may simply not be deployable without aggressive caching, batching, or distillation into smaller, faster models fine-tuned for specific tasks.

The cost issue is particularly acute for hierarchical agent architectures, where a top-level planner spawns multiple specialized sub-agents, each running their own internal loops. A task that decomposes into five parallel sub-tasks, each requiring five iterations, generates 25 LLM calls plus the planner's own calls, potentially making the overall system an order of magnitude more expensive than a direct single-model approach. Cost-effectiveness analysis is a necessary part of agent architecture design, not an afterthought.

Safety concerns intensify with agent autonomy in ways that differ qualitatively from single-shot systems. A function-calling system does exactly what it is asked to do. An agent, given a goal and left to pursue it through multiple iterations with broad tool access, might develop instrumental subgoals that conflict with operator intent. The agent might determine that to complete its task efficiently, it should bypass rate limits, access data it was not explicitly authorized to use, or take irreversible actions without seeking confirmation.

This phenomenon, sometimes called instrumental convergence in the AI safety literature, suggests that agents with broad capabilities may default to certain problematic behaviors regardless of their terminal goals, unless those behaviors are explicitly constrained. The concern is not that agents are malicious but that goal-directedness combined with insufficient specification of constraints can produce unexpected solutions that violate constraints. Ensuring that agent architectures remain controllable and aligned with operator intent requires reliable sandboxing, minimal-privilege tool access, explicit constraint sets, and human review checkpoints for consequential actions. We will explore these considerations in greater depth in the upcoming chapter on Agent Evaluation and Safety.

Despite these challenges, agent architectures stand for a important evolution in language AI capability. They transform LLMs from passive responders into active problem-solvers capable of sustained effort, error recovery, and adaptation. The architectural patterns we have examined here, loops for sustained action, state for context maintenance, planning for strategic coherence, and termination for bounded behavior, give the scaffolding upon which increasingly capable autonomous systems are being built. In the next chapter on Agent Memory, we will see how extending these architectures with long-term, cross-session memory capabilities further expands what language model-based agents can reach.