Michael Brenndoerfer | Data & AI, Private Equity, Technology

Michael Brenndoerfer

ReAct Pattern: Interleaving Reasoning and Action for LLM Agents › Context Length and Token Growth

Michael BrenndoerferFebruary 3, 20263 min read

Language AI HandbookAll chapters

Part 48 of 67Tool Use and Agents

Chapter 365ReAct Pattern: Interleaving Reasoning and Action for LLM Agents › Context Length and Token Growth

Reading section 7 of 14 · Context Length and Token Growth · 3 minSection 7 of 14 · 3 min

Section 7 of 14 · 3 min read

ReAct Pattern: Interleaving Reasoning and Action for LLM Agents › Context Length and Token GrowthOne section · Sections in this chapter7 / 14 ↓

Sections in this chapter14

Reading tools Definitions & inline links

Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.

Article links

Make inline references clickable

One of the most practically important aspects of ReAct is that context grows with each iteration. The prompt at step $n$ contains:

|P_n| = |P_{\text{system}}| + \sum_{i=1}^{n-1} (|t_i| + |a_i| + |o_i|)

where $|P_{\text{system}}|$ is the length of the system prompt (including few-shot examples), and each iteration adds tokens for a thought and action plus the resulting observation. For typical ReAct setups with roughly 350 tokens for the system prompt and roughly 280 tokens per iteration (about 80 for the thought, 40 for the action, and 160 for the observation), the context length at step $n$ is approximately:

|P_n| \approx 350 + 280 \cdot (n - 1)

This linear growth has real consequences. Modern models support context windows of 8k, 32k, 128k, or even 1M tokens, but most API calls are billed per token, and latency increases with context length. An agent running 20 iterations with verbose observations could easily consume 10,000 tokens per LLM call, and it makes 20 such calls, for a total of 200,000 tokens for a single user question. At typical API pricing, this adds up quickly.

Out[10]:

Visualization

Line chart showing cumulative token count rising linearly across 12 ReAct iterations, with a dashed red line at 4,096 tokens. — Cumulative context length growth across ReAct iterations, assuming 350 tokens for the system prompt and 280 tokens per iteration. The red dashed line marks a 4,096-token context window limit, showing that agents running more than about 13 iterations will exceed this boundary. Modern models with larger context windows shift this limit right but do not eliminate the linear growth problem.

The chart shows that even with a modest 4k context window, the agent starts hitting limits around iteration 13. With modern 128k context windows, this is less immediately problematic, but the cost and latency implications remain. Several strategies address this:

Trajectory truncation: Drop the oldest few iterations when the context approaches a threshold. This loses some historical context but keeps the agent running.
Observation summarization: Instead of storing the full observation, store a one-sentence summary. This is lossy but dramatically reduces token usage.
Selective retention: Keep all thoughts and actions, but only keep observations when they contributed new information. Redundant or failed observations are discarded.
External memory: For very long sessions, store the trajectory in a database and retrieve only the most relevant past steps using semantic similarity search.

The right strategy depends on the task. For short research queries (5-10 steps), the raw trajectory works fine. For complex research or coding tasks that might run dozens of iterations, some form of compression becomes necessary.