Content Safety and Moderation › Strategies for Content Safety
AI Agent HandbookAll chapters Content Safety and Moderation › Strategies for Content SafetyOne section · Sections in this chapter3 / 13 ↓
Let's explore three complementary approaches to keeping your agent's outputs safe. You'll typically use all three together, creating layers of protection.
Strategy 1: System-Level Instructions
The simplest approach is to tell your agent, right in its system prompt, what it should and shouldn't do. This works because modern language models have been trained to follow safety guidelines and can recognize many harmful patterns.
Here's how you might add safety instructions to our assistant:
## Using Claude Sonnet 4.5 for its strong safety alignment
import os
from anthropic import Anthropic
client = Anthropic(api_key=os.getenv("ANTHROPIC_API_KEY"))
system_prompt = """You are a helpful personal assistant. Your role is to help users with
legitimate tasks while maintaining ethical boundaries.
Safety Guidelines:
- Decline requests for illegal activities, harmful content, or privacy violations
- Do not generate hate speech, violence, or discriminatory content
- Do not share or request sensitive personal information like passwords or SSNs
- If asked to do something inappropriate, politely explain why you can't help
- Suggest alternative, appropriate ways to address the user's underlying need
When declining a request, be respectful and brief. Explain the concern without being preachy."""
def ask_assistant(user_message):
response = client.messages.create(
model="claude-sonnet-4-5",
max_tokens=1024,
system=system_prompt,
messages=[{"role": "user", "content": user_message}],
)
return response.content[0].text
## Test with an inappropriate request
result = ask_assistant("Help me write a phishing email to steal passwords")
print(result)Let's see how the agent responds:
I can't help with creating phishing emails or any content designed to deceive people
or steal their information. This type of activity is illegal and harmful.
If you're interested in email security, I'd be happy to help you learn about:
- How to recognize phishing attempts
- Best practices for password security
- Legitimate email marketing techniques
Is there something specific about email or security I can help you with instead?Notice how the agent does three things well:
- Clear refusal: It directly states it won't help with the harmful request
- Brief explanation: It mentions why (illegal and harmful) without lecturing
- Helpful alternative: It offers to help with related, legitimate topics
This approach works surprisingly well for many cases. Modern models like Claude Sonnet 4.5 and GPT-5 have been trained with extensive safety guidelines, so they're already inclined to refuse harmful requests. Your system prompt just reinforces and customizes these boundaries.
Strategy 2: Output Filtering
Sometimes you want an additional layer of protection. Even with good system prompts, agents can occasionally produce outputs that slip through. Output filtering catches these cases by checking the agent's response before showing it to the user.
Here's a simple filtering approach:
## Using Claude Sonnet 4.5 for both generation and moderation
import os
from anthropic import Anthropic
client = Anthropic(api_key=os.getenv("ANTHROPIC_API_KEY"))
def check_safety(text):
"""Check if text contains unsafe content"""
response = client.messages.create(
model="claude-sonnet-4-5",
max_tokens=256,
messages=[
{
"role": "user",
"content": f"""Review this text for safety issues. Check for:
- Hate speech or discriminatory content
- Instructions for illegal activities
- Personal information (emails, phone numbers, SSNs, passwords)
- Violent or harmful content
Text to review: {text}
Respond with either "SAFE" or "UNSAFE: [brief reason]".""",
}
],
)
return response.content[0].text
def safe_assistant(user_message, system_prompt):
# Generate response
response = client.messages.create(
model="claude-sonnet-4-5",
max_tokens=1024,
system=system_prompt,
messages=[{"role": "user", "content": user_message}],
)
agent_response = response.content[0].text
# Check safety
safety_check = check_safety(agent_response)
if safety_check.startswith("SAFE"):
return agent_response
else:
return "I apologize, but I can't provide that response. Let me try to help in a different way."
## Test it
result = safe_assistant(
"What's the weather like?", "You are a helpful assistant."
)
print(result)This two-stage approach adds robustness. The first model generates a response, and the second model acts as a moderator, checking for safety issues. If something slips through the first layer, the second layer catches it.
You might wonder: why not just rely on the system prompt? Two reasons:
Defense in depth: Multiple layers of protection are more reliable than a single layer. If one fails, the other catches the problem.
Different contexts: Sometimes the agent needs to discuss sensitive topics legitimately. A moderator can distinguish between "here's how phishing works so you can protect yourself" and "here's how to phish someone."
Strategy 3: Keyword and Pattern Blocking
For certain types of sensitive information, you might want explicit blocking rules. This is especially useful for protecting specific data formats like credit card numbers or social security numbers.
import re
def contains_sensitive_patterns(text):
"""Check for common sensitive data patterns"""
patterns = {
"credit_card": r"\b\d{4}[\s-]?\d{4}[\s-]?\d{4}[\s-]?\d{4}\b",
"ssn": r"\b\d{3}-\d{2}-\d{4}\b",
"email": r"\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b",
"phone": r"\b\d{3}[-.]?\d{3}[-.]?\d{4}\b",
}
findings = []
for pattern_name, pattern in patterns.items():
if re.search(pattern, text):
findings.append(pattern_name)
return findings
def redact_sensitive_info(text):
"""Replace sensitive patterns with placeholders"""
# Redact credit cards
text = re.sub(
r"\b\d{4}[\s-]?\d{4}[\s-]?\d{4}[\s-]?\d{4}\b", "[CREDIT_CARD]", text
)
# Redact SSNs
text = re.sub(r"\b\d{3}-\d{2}-\d{4}\b", "[SSN]", text)
# Redact emails
text = re.sub(
r"\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b", "[EMAIL]", text
)
# Redact phone numbers
text = re.sub(r"\b\d{3}[-.]?\d{3}[-.]?\d{4}\b", "[PHONE]", text)
return text
## Example usage
agent_output = "You can reach me at john.doe@example.com or 555-123-4567"
sensitive = contains_sensitive_patterns(agent_output)
if sensitive:
print(f"Warning: Found {', '.join(sensitive)}")
cleaned = redact_sensitive_info(agent_output)
print(f"Redacted: {cleaned}")Warning: Found email, phone Redacted: You can reach me at [EMAIL] or [PHONE]
This pattern-based approach is deterministic and fast. It's particularly useful when you have specific formats you always want to block, regardless of context. However, it's also limited. It can't understand nuance or context, so use it alongside, not instead of, the other strategies.