Michael Brenndoerfer | Data & AI, Private Equity, Technology

Michael Brenndoerfer

Short-Term Conversation Memory › Choosing a Memory Strategy

Michael BrenndoerferJuly 1, 20252 min read

AI Agent HandbookAll chapters

Part 6 of 15Memory and Retrieval - How Agents Remember

Chapter 15Short-Term Conversation Memory › Choosing a Memory Strategy

Reading section 11 of 15 · Choosing a Memory Strategy · 2 minSection 11 of 15 · 2 min

Section 11 of 15 · 2 min read

Short-Term Conversation Memory › Choosing a Memory StrategyOne section · Sections in this chapter11 / 15 ↓

Sections in this chapter15

Which approach should you use? It depends on your use case:

Full history (no limits):

Best for: Short conversations, when cost isn't a concern
Pros: Perfect memory, no information loss
Cons: Expensive for long conversations, can hit context limits

Sliding window:

Best for: Casual chat, when only recent context matters
Pros: Simple, predictable cost, never hits context limits
Cons: Forgets older information completely

Summarization:

Best for: Long conversations where older context matters
Pros: Preserves important information, manages token count
Cons: More complex, costs extra for summarization, might lose details

For our personal assistant, a hybrid approach often works best:

In[15]:

Code

class SmartConversationalAgent:
    """Agent with intelligent memory management."""
    
    def __init__(self,
                 window_size: int = 30,
                 summarize_threshold: int = 50):
        self.client = Anthropic(api_key=os.getenv("ANTHROPIC_API_KEY"))
        self.conversation_history = []
        self.window_size = window_size
        self.summarize_threshold = summarize_threshold
        self.summary = None
    
    def chat(self, user_message: str) -> str:
        """Chat with smart memory management."""
        # Add message
        self.conversation_history.append({
            "role": "user",
            "content": user_message
        })
        
        # Manage memory
        if len(self.conversation_history) > self.summarize_threshold:
            # Summarize old messages
            old_messages = self.conversation_history[:-self.window_size]
            self.summary = self._summarize(old_messages)
            self.conversation_history = self.conversation_history[-self.window_size:]
        
        # Build context
        messages = []
        if self.summary:
            messages.append({
                "role": "user",
                "content": f"[Context from earlier: {self.summary}]"
            })
        messages.extend(self.conversation_history)
        
        # Get response
        response = self.client.messages.create(
            model="claude-sonnet-4-5",
            max_tokens=1024,
            messages=messages
        )
        
        assistant_message = response.content[0].text
        self.conversation_history.append({
            "role": "assistant",
            "content": assistant_message
        })
        
        return assistant_message

This combines both strategies: keep recent messages in full, summarize older ones.