Agent Architectures: Control Loops, State & Planning › State Management

State is the memory of an agent system: the accumulated information that persists across loop iterations and lets coherent, context-aware behavior. Without effective state management, each loop iteration would be an isolated, stateless exchange, incapable of maintaining continuity or learning from previous actions. The challenge is particularly acute in LLM-based agents because, unlike traditional software systems that maintain arbitrary data structures in memory, agents must compress their state into the limited context window of the language model or manage it externally through retrieval systems.

The problem of state management becomes visible when you trace what an agent needs to know at each iteration. It needs the original goal, the actions it has already taken, the results of those actions, facts it has discovered along the way, the current plan, and any error conditions encountered. All of this information must be available during the reasoning step, or the agent loses coherence. Managing these different streams of information and keeping them organized and prioritized within the bounds of what the model can process is the central challenge of state management.

Agent architectures typically manage several categories of state, each serving distinct cognitive functions.

Conversation history maintains the chronological sequence of observations and thoughts followed by actions. This gives the LLM with context about what has already been attempted and what has been discovered. As we discussed in Part XXVIII on GPT Architecture, transformers process context through attention mechanisms, meaning the entire conversation history influences each new token generation. However, raw conversation history can become lengthy and redundant, containing facts alongside the procedural noise of how those facts were obtained: failed attempts, dead ends, and repeated queries that eventually yielded results. The agent must balance the need for complete context against the attention costs and context window limits of processing lengthy histories.

Working memory holds structured information extracted from observations: facts retrieved from knowledge bases, intermediate calculation results, parsed user intents, and active hypotheses. Unlike raw conversation history, working memory is often summarized or structured to reduce token consumption and highlight salient information. Think of working memory as the agent's scratchpad, where it keeps the current hypotheses, active goals, and necessary facts that demand immediate attention. This is analogous to human working memory, which has limited capacity but fast access, holding the information currently needed for the task at hand. A research agent might maintain working memory as a dictionary mapping topic headings to bullet-point summaries of what has been learned about each.

Episodic memory stores past experiences for long-term retrieval, persisting beyond the boundaries of a single conversation. While conversation history is linear and ephemeral (cleared between sessions), episodic memory allows agents to recall relevant previous interactions when encountering similar situations. This might include remembering that a particular database query pattern failed in a past session, or that a specific user prefers concise answers without technical jargon. We will explore memory architectures in greater detail in the upcoming chapter on Agent Memory, but episodic memory is worth understanding here as the mechanism that allows agents to learn from experience across conversations rather than starting fresh each time.

Tool state tracks the status of external systems: database connections, API session tokens, file handles, or the current position in paginated search results. This state ensures that tools maintain consistency across multiple invocations within a session. For example, if an agent is iterating through a large dataset using cursor-based pagination, the cursor position is needed state that must persist between tool calls. If this state is lost, the agent would restart from the beginning of the dataset on every call, potentially missing records or processing duplicates. Similarly, maintaining authentication tokens across API calls prevents the overhead of re-authenticating with each iteration, which could be both slow and potentially trigger rate limits.

The finite context window of LLMs creates a basic tension: agents need complete state to make informed decisions, but they can only attend to limited amounts of text. Several strategies address this constraint, each with different tradeoffs between completeness and efficiency.

Truncation with summarization discards older conversation turns while preserving a condensed summary. The agent might maintain the full context of recent interactions but compress distant history into key facts and decisions. For example, after ten iterations of web searching, the agent might summarize the first five iterations as a brief paragraph capturing the main sources found and their key claims, while keeping the last five iterations in full detail. This approach risks losing nuance (a seemingly minor detail from iteration two might become relevant at iteration eleven) but preserves the most immediately relevant information within a manageable token budget.

Hierarchical state organizes information at different granularities. Immediate context (the last few turns) is preserved verbatim; short-term summaries capture the current task phase; long-term memory stores extracted knowledge in structured form. This mirrors human cognitive architectures where working memory and long-term memory serve different functions, with information flowing between them through attention and consolidation. In practice, this might mean the agent keeps the last three tool calls in full detail, maintains a paragraph summary of the current subtask progress, and stores key facts in a structured knowledge graph. Each tier serves a different temporal horizon of reasoning: the immediate context supports fine-grained decision making, while the long-term store supports goal-level planning.

External state stores move state outside the LLM's context window into databases, vector stores, or structured files. The agent then uses retrieval mechanisms to pull relevant state into context on demand, building on the dense retrieval concepts from our discussion of retrieval-augmented generation. This approach allows agents to maintain effectively unlimited state while keeping context windows manageable. A legal research agent might store all discovered case law in a vector database, querying it for relevant precedents only when drafting specific arguments, rather than keeping every case in the active context. The tradeoff is retrieval quality: if the retrieval mechanism fails to retrieve a relevant piece of state, the agent proceeds without it, potentially making decisions inconsistent with prior work.

Maintaining consistent state across operations presents significant challenges, particularly when agents execute long-running tools or parallel subtasks where state updates may arrive out of order or fail entirely. Reliable architectures implement several strategies to ensure coherence.

Atomic updates ensure that state changes occur as complete transactions, preventing partial updates that leave the agent in inconsistent configurations. If an agent is updating both a task list and a knowledge base simultaneously, both updates should succeed or both should fail. Allowing one to succeed while the other fails creates an inconsistency where completed tasks are not reflected in the knowledge store, or conversely, where knowledge is updated for tasks that were never marked complete.

Conflict resolution gives rules for reconciling conflicting changes when concurrent processes modify shared state. If two sub-agents simultaneously update the same working memory key with different values, the architecture needs a clear policy for determining which update takes precedence. Common strategies include timestamp-based ordering (most recent write wins), priority-based resolution (the higher-priority agent wins), and merge operations (both values are combined, for example by appending to a list rather than overwriting).

Checkpointing creates periodic snapshots of agent state that let recovery from failures without losing all progress. If an agent crashes midway through a complex task, checkpointing allows it to resume from the most recent checkpoint rather than starting over from the beginning. This is especially important for long-running agents where the cost of restarting from scratch, in both time and API tokens, is prohibitive.

State management directly affects an agent's ability to reason coherently. An agent operating on inconsistent state may generate contradictory plans, repeat actions it has already taken, or fail to recognize when a goal has been achieved. Investing in reliable state management is investing in the agent's cognitive reliability.