Content Safety and Moderation › Combining Strategies: A Complete Safety System

Michael BrenndoerferAugust 12, 20252 min read
AI Agent HandbookAll chapters
Part 13 of 15Safety and Governance - Keeping the Agent Aligned
Chapter 36Content Safety and Moderation › Combining Strategies: A Complete Safety System
Reading section 6 of 13 · Combining Strategies: A Complete Safety System · 2 minSection 6 of 13 · 2 min

Book contents

Section 6 of 13 · 2 min read

Content Safety and Moderation › Combining Strategies: A Complete Safety SystemOne section · Sections in this chapter6 / 13 ↓

In practice, you'll use all these strategies together. Here's how they fit together in our personal assistant:

In[2]:
Code
## Using Claude Sonnet 4.5 for comprehensive safety handling
import os
from anthropic import Anthropic
import re

client = Anthropic(api_key=os.getenv("ANTHROPIC_API_KEY"))


class SafeAssistant:
    def __init__(self):
        self.system_prompt = """You are a helpful personal assistant with strong ethical boundaries.
        
Safety Guidelines:
- Decline illegal, harmful, or deceptive requests politely
- Do not share sensitive information like passwords or financial data
- When declining, acknowledge the goal and offer appropriate alternatives
- Be respectful and helpful while maintaining boundaries"""

    def contains_sensitive_patterns(self, text):
        """Check for sensitive data patterns"""
        patterns = {
            "credit_card": r"\b\d{4}[\s-]?\d{4}[\s-]?\d{4}[\s-]?\d{4}\b",
            "ssn": r"\b\d{3}-\d{2}-\d{4}\b",
            "password": r"\b[Pp]assword:\s*\S+",
        }

        for pattern_name, pattern in patterns.items():
            if re.search(pattern, text):
                return True, pattern_name
        return False, None

    def moderate_content(self, text):
        """Check if content is safe using another model call"""
        response = client.messages.create(
            model="claude-sonnet-4-5",
            max_tokens=128,
            messages=[
                {
                    "role": "user",
                    "content": f"""Is this response safe to show a user? Check for:
- Harmful instructions
- Hate speech
- Privacy violations

Response: {text}

Answer with just "SAFE" or "UNSAFE".""",
                }
            ],
        )
        return response.content[0].text.strip() == "SAFE"

    def respond(self, user_message):
        """Generate a safe response with multiple protection layers"""
        # Layer 1: Generate response with safety-aware system prompt
        response = client.messages.create(
            model="claude-sonnet-4-5",
            max_tokens=1024,
            system=self.system_prompt,
            messages=[{"role": "user", "content": user_message}],
        )

        agent_response = response.content[0].text

        # Layer 2: Check for sensitive patterns
        has_sensitive, pattern_type = self.contains_sensitive_patterns(
            agent_response
        )
        if has_sensitive:
            return f"I apologize, but my response contained sensitive information ({pattern_type}). Let me help you in a safer way."

        # Layer 3: Content moderation
        if not self.moderate_content(agent_response):
            return "I apologize, but I need to reconsider my response. How else can I help you?"

        return agent_response


## Use the safe assistant
assistant = SafeAssistant()

## Test various scenarios
test_cases = [
    "What's the weather like today?",  # Normal request
    "Help me write a threatening message",  # Harmful request
    "My password is secret123, can you remember it?",  # Sensitive info
]

for test in test_cases:
    print(f"User: {test}")
    print(f"Assistant: {assistant.respond(test)}\n")

This complete system has three layers of protection:

  1. System prompt: Teaches the agent to refuse inappropriate requests
  2. Pattern detection: Catches specific sensitive data formats
  3. Content moderation: Double-checks outputs for safety issues

Each layer catches different types of problems. The system prompt handles most cases. Pattern detection catches specific formats that might slip through. Content moderation provides a final safety net.