Long-Term Knowledge Storage and Retrieval › Integrating Long-Term Memory with Our Assistant

Michael BrenndoerferJuly 3, 20253 min read
AI Agent HandbookAll chapters
Part 6 of 15Memory and Retrieval - How Agents Remember
Chapter 16Long-Term Knowledge Storage and Retrieval › Integrating Long-Term Memory with Our Assistant
Reading section 6 of 13 · Integrating Long-Term Memory with Our Assistant · 3 minSection 6 of 13 · 3 min

Book contents

AI Agent Handbook15 parts · 44 chaptersView the whole book →
Active chapter 16 of 44

Section 6 of 13 · 3 min read

Long-Term Knowledge Storage and Retrieval › Integrating Long-Term Memory with Our AssistantOne section · Sections in this chapter6 / 13 ↓
Sections in this chapter13
Reading tools Definitions & inline links

Toggle tooltip visibility. Hover over underlined terms for instant definitions.

Article links

Make inline references clickable

Now let's connect this to our personal assistant. We want the assistant to:

  1. Recognize when you're telling it something to remember
  2. Store that information in long-term memory
  3. Retrieve relevant information when answering questions
  4. Combine retrieved knowledge with its language model capabilities

Here's how this works with Claude Sonnet 4.5 (Example: Claude Sonnet 4.5):

In[6]:
Code
import os
from anthropic import Anthropic


class AssistantWithMemory:
    def __init__(self):
        self.client = Anthropic(api_key=os.getenv("ANTHROPIC_API_KEY"))
        # Using Claude Sonnet 4.5 for its superior agent reasoning capabilities
        self.model = "claude-sonnet-4-5"
        self.memory = VectorMemory()
        self.conversation_history = []

    def process_message(self, user_message):
        """Process a user message, using memory when relevant"""
        # First, check if this is a request to remember something
        if self._is_memory_request(user_message):
            return self._store_memory(user_message)

        # Search for relevant facts
        relevant_facts = self.memory.search(user_message, top_k=3)

        # Build context with conversation history and retrieved facts
        context = self._build_context(relevant_facts)

        # Get response from Claude
        response = self.client.messages.create(
            model=self.model,
            max_tokens=1024,
            system=context,
            messages=self.conversation_history
            + [{"role": "user", "content": user_message}],
        )

        assistant_message = response.content[0].text

        # Update conversation history
        self.conversation_history.append(
            {"role": "user", "content": user_message}
        )
        self.conversation_history.append(
            {"role": "assistant", "content": assistant_message}
        )

        return assistant_message

    def _is_memory_request(self, message):
        """Detect if user wants to store information"""
        memory_keywords = [
            "remember",
            "store",
            "save",
            "keep in mind",
            "note that",
        ]
        return any(keyword in message.lower() for keyword in memory_keywords)

    def _store_memory(self, message):
        """Extract and store information from user message"""
        # Use Claude to extract the fact to remember
        response = self.client.messages.create(
            model=self.model,
            max_tokens=256,
            system="Extract the key fact the user wants you to remember. Return only the fact as a clear statement.",
            messages=[{"role": "user", "content": message}],
        )

        fact = response.content[0].text
        self.memory.add_fact(fact)
        return f"Got it! I'll remember that {fact.lower()}"

    def _build_context(self, relevant_facts):
        """Build system context with retrieved facts"""
        if not relevant_facts or relevant_facts == [
            "No information stored yet."
        ]:
            return "You are a helpful personal assistant."

        facts_text = "\n".join(f"- {fact}" for fact in relevant_facts)
        return f"""You are a helpful personal assistant. You have access to the following information about the user:

{facts_text}

Use this information when relevant to provide personalized responses."""


## Example conversation
assistant = AssistantWithMemory()

print(assistant.process_message("Remember that my birthday is July 20"))
## Output: Got it! I'll remember that your birthday is july 20

print(assistant.process_message("Remember that I'm allergic to peanuts"))
## Output: Got it! I'll remember that you're allergic to peanuts

print(
    assistant.process_message("What should I be careful about when eating out?")
)
## Output: Based on what I know, you should be careful about peanuts since you're
## allergic to them. When eating out, make sure to inform the restaurant staff about
## your peanut allergy and ask about ingredients in dishes...
Out[6]:
Console
Loading weights:   0%|          | 0/103 [00:00<?, ?it/s]
TypeError: "Could not resolve authentication method. Expected either api_key or auth_token to be set. Or for one of the `X-Api-Key` or `Authorization` headers to be explicitly omitted"
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[6], line 89
     86 ## Example conversation
     87 assistant = AssistantWithMemory()
---> 89 print(assistant.process_message("Remember that my birthday is July 20"))
     90 ## Output: Got it! I'll remember that your birthday is july 20
     92 print(assistant.process_message("Remember that I'm allergic to peanuts"))

Cell In[6], line 17, in AssistantWithMemory.process_message(self, user_message)
     15 # First, check if this is a request to remember something
     16 if self._is_memory_request(user_message):
---> 17     return self._store_memory(user_message)
     19 # Search for relevant facts
     20 relevant_facts = self.memory.search(user_message, top_k=3)

Cell In[6], line 60, in AssistantWithMemory._store_memory(self, message)
     58 """Extract and store information from user message"""
     59 # Use Claude to extract the fact to remember
---> 60 response = self.client.messages.create(
     61     model=self.model,
     62     max_tokens=256,
     63     system="Extract the key fact the user wants you to remember. Return only the fact as a clear statement.",
     64     messages=[{"role": "user", "content": message}],
     65 )
     67 fact = response.content[0].text
     68 self.memory.add_fact(fact)

File /private/tmp/mb-ai-style-scanner/.venv/lib/python3.11/site-packages/anthropic/_utils/_utils.py:282, in required_args.<locals>.inner.<locals>.wrapper(*args, **kwargs)
    280             msg = f"Missing required argument: {quote(missing[0])}"
    281     raise TypeError(msg)
--> 282 return func(*args, **kwargs)

File /private/tmp/mb-ai-style-scanner/.venv/lib/python3.11/site-packages/anthropic/resources/messages/messages.py:996, in Messages.create(self, max_tokens, messages, model, cache_control, container, inference_geo, metadata, output_config, service_tier, stop_sequences, stream, system, temperature, thinking, tool_choice, tools, top_k, top_p, extra_headers, extra_query, extra_body, timeout)
    989 if model in MODELS_TO_WARN_WITH_THINKING_ENABLED and thinking and thinking["type"] == "enabled":
    990     warnings.warn(
    991         f"Using Claude with {model} and 'thinking.type=enabled' is deprecated. Use 'thinking.type=adaptive' instead which results in better model performance in our testing: https://platform.claude.com/docs/en/build-with-claude/adaptive-thinking",
    992         UserWarning,
    993         stacklevel=3,
    994     )
--> 996 return self._post(
    997     "/v1/messages",
    998     body=maybe_transform(
    999         {
   1000             "max_tokens": max_tokens,
   1001             "messages": messages,
   1002             "model": model,
   1003             "cache_control": cache_control,
   1004             "container": container,
   1005             "inference_geo": inference_geo,
   1006             "metadata": metadata,
   1007             "output_config": output_config,
   1008             "service_tier": service_tier,
   1009             "stop_sequences": stop_sequences,
   1010             "stream": stream,
   1011             "system": system,
   1012             "temperature": temperature,
   1013             "thinking": thinking,
   1014             "tool_choice": tool_choice,
   1015             "tools": tools,
   1016             "top_k": top_k,
   1017             "top_p": top_p,
   1018         },
   1019         message_create_params.MessageCreateParamsStreaming
   1020         if stream
   1021         else message_create_params.MessageCreateParamsNonStreaming,
   1022     ),
   1023     options=make_request_options(
   1024         extra_headers=extra_headers, extra_query=extra_query, extra_body=extra_body, timeout=timeout
   1025     ),
   1026     cast_to=Message,
   1027     stream=stream or False,
   1028     stream_cls=Stream[RawMessageStreamEvent],
   1029 )

File /private/tmp/mb-ai-style-scanner/.venv/lib/python3.11/site-packages/anthropic/_base_client.py:1364, in SyncAPIClient.post(self, path, cast_to, body, content, options, files, stream, stream_cls)
   1355     warnings.warn(
   1356         "Passing raw bytes as `body` is deprecated and will be removed in a future version. "
   1357         "Please pass raw bytes via the `content` parameter instead.",
   1358         DeprecationWarning,
   1359         stacklevel=2,
   1360     )
   1361 opts = FinalRequestOptions.construct(
   1362     method="post", url=path, json_data=body, content=content, files=to_httpx_files(files), **options
   1363 )
-> 1364 return cast(ResponseT, self.request(cast_to, opts, stream=stream, stream_cls=stream_cls))

File /private/tmp/mb-ai-style-scanner/.venv/lib/python3.11/site-packages/anthropic/_base_client.py:1058, in SyncAPIClient.request(self, cast_to, options, stream, stream_cls)
   1055 options = self._prepare_options(options)
   1057 remaining_retries = max_retries - retries_taken
-> 1058 request = self._build_request(options, retries_taken=retries_taken)
   1059 self._prepare_request(request)
   1061 kwargs: HttpxSendArgs = {}

File /private/tmp/mb-ai-style-scanner/.venv/lib/python3.11/site-packages/anthropic/_base_client.py:521, in BaseClient._build_request(self, options, retries_taken)
    518     else:
    519         raise RuntimeError(f"Unexpected JSON data type, {type(json_data)}, cannot merge with `extra_body`")
--> 521 headers = self._build_headers(options, retries_taken=retries_taken)
    522 params = _merge_mappings(self.default_query, options.params)
    523 content_type = headers.get("Content-Type")

File /private/tmp/mb-ai-style-scanner/.venv/lib/python3.11/site-packages/anthropic/_base_client.py:451, in BaseClient._build_headers(self, options, retries_taken)
    441 custom_headers = options.headers or {}
    442 headers_dict = _merge_mappings(
    443     {
    444         "x-stainless-timeout": str(options.timeout.read)
   (...)    449     custom_headers,
    450 )
--> 451 self._validate_headers(headers_dict, custom_headers)
    453 # headers are case-insensitive while dictionaries are not.
    454 headers = httpx.Headers(headers_dict)

File /private/tmp/mb-ai-style-scanner/.venv/lib/python3.11/site-packages/anthropic/_client.py:196, in Anthropic._validate_headers(self, headers, custom_headers)
    193 if headers.get("Authorization") or isinstance(custom_headers.get("Authorization"), Omit):
    194     return
--> 196 raise TypeError(
    197     '"Could not resolve authentication method. Expected either api_key or auth_token to be set. Or for one of the `X-Api-Key` or `Authorization` headers to be explicitly omitted"'
    198 )

TypeError: "Could not resolve authentication method. Expected either api_key or auth_token to be set. Or for one of the `X-Api-Key` or `Authorization` headers to be explicitly omitted"

Let's trace through what happens when you ask "What should I be careful about when eating out?":

  1. The assistant searches its vector memory for relevant facts
  2. It finds "You're allergic to peanuts" as highly relevant
  3. It includes this fact in the system context when calling Claude
  4. Claude uses this information to provide a personalized, helpful response

The assistant now has a persistent memory that survives across sessions. You can close the program, restart it days later, and it will still remember your allergy.