Content Safety and Moderation › Protecting Privacy in Responses

Michael BrenndoerferAugust 12, 20253 min read
AI Agent HandbookAll chapters
Part 13 of 15Safety and Governance - Keeping the Agent Aligned
Chapter 36Content Safety and Moderation › Protecting Privacy in Responses
Reading section 5 of 13 · Protecting Privacy in Responses · 3 minSection 5 of 13 · 3 min

Book contents

Section 5 of 13 · 3 min read

Content Safety and Moderation › Protecting Privacy in ResponsesOne section · Sections in this chapter5 / 13 ↓

Your agent might have access to sensitive information through its memory or tools. It needs to know what information is safe to share and what should stay private.

Defining Privacy Boundaries

Start by categorizing information:

Always safe to share:

  • General knowledge
  • Public information
  • Information the user explicitly provided in the current conversation

Requires context:

  • Information from the user's past conversations
  • Data retrieved from tools
  • Aggregated or summarized information

Never share:

  • Authentication credentials
  • Financial account numbers
  • Social security numbers or government IDs
  • Medical information (unless explicitly requested by the user)

You can encode these rules in your system prompt:

In[2]:
Code
## Using Claude Sonnet 4.5 for privacy-aware responses
import os
from anthropic import Anthropic

client = Anthropic(api_key=os.getenv("ANTHROPIC_API_KEY"))

system_prompt = """You are a personal assistant with access to user information. 
Follow these privacy rules strictly:

NEVER share:
- Passwords or authentication credentials
- Full credit card or bank account numbers
- Social security numbers or government IDs
- Exact addresses or precise location data
- Medical information

You MAY share:
- Information the user just told you in this conversation
- General facts and public knowledge
- Summaries that don't reveal sensitive details

When asked about sensitive information:

1. Confirm you have the information but cannot share it directly
2. Offer to help in a privacy-safe way
3. Suggest the user access the information directly if needed

Example: If asked for a password, respond: "I don't share passwords for security reasons. 
I can help you reset it or guide you to where it's stored securely."
"""


def privacy_safe_response(user_message, user_data=None):
    # In a real system, user_data might come from memory or a database
    context = f"User data available: {user_data}" if user_data else ""

    response = client.messages.create(
        model="claude-sonnet-4-5",
        max_tokens=1024,
        system=f"{system_prompt}\n\n{context}",
        messages=[{"role": "user", "content": user_message}],
    )
    return response.content[0].text


## Test privacy handling
user_data = {
    "name": "Alice",
    "email": "alice@example.com",
    "password": "secret123",  # Should never be shared
    "credit_card": "4532-1234-5678-9010",  # Should never be shared
}

result = privacy_safe_response("What's my email and password?", user_data)
print(result)

The agent should respond something like:

Your email is alice@example.com. 

I don't share passwords for security reasons. If you need to access your password, 
I recommend checking your password manager or using the "forgot password" feature 
to reset it securely.

Notice how it shares the email (which is relatively low-risk and the user asked for it) but refuses to share the password. It then gives the user safer alternatives.