Perception and Action › Perception Limitations and Action Constraints
AI Agent HandbookAll chapters Perception and Action › Perception Limitations and Action ConstraintsOne section · Sections in this chapter6 / 10 ↓
Our assistant doesn't perceive everything, and it can't do everything. Understanding these boundaries is crucial for building reliable agents.
Perception Limitations
Partial observability: The agent can't see everything in its environment. It only perceives what it explicitly checks. If you have a calendar event but the agent doesn't query the calendar, it won't know about the event.
Noisy perception: Sometimes the agent misinterprets what it perceives. A user's ambiguous question might be understood incorrectly, or a tool might return incomplete data.
Delayed perception: The agent perceives information at specific moments. It doesn't continuously monitor its environment. Between cycles, things might change without the agent knowing.
Action Constraints
Limited capabilities: The agent can only take actions it has tools for. If there's no email tool, it can't send emails, no matter how much it wants to.
Permission boundaries: Even with tools available, the agent might not have permission to use them in all situations. We might restrict it from deleting files or making purchases without confirmation.
Action failures: Tools can fail. APIs go down, databases become unavailable, or operations time out. The agent must handle these failures gracefully.
Handling Limitations in Code (Claude Sonnet 4.5)
Here's how to build robustness into the perception-action cycle:
class RobustAgent:
def __init__(self, available_tools):
self.tools = available_tools
self.perception_history = []
def safe_perceive(self, source, data):
"""Perceive with error handling"""
try:
perception = {
"source": source,
"data": data,
"success": True
}
self.perception_history.append(perception)
return perception
except Exception as e:
return {
"source": source,
"error": str(e),
"success": False
}
def safe_act(self, action_type, params):
"""Act with constraints and error handling"""
# Check if action is allowed
if action_type not in self.tools:
return {
"success": False,
"error": f"Tool {action_type} not available"
}
# Try to execute action
try:
result = self.tools[action_type](params)
return {"success": True, "result": result}
except Exception as e:
return {
"success": False,
"error": f"Action failed: {str(e)}"
}
def run_with_fallback(self, user_input):
"""Run cycle with fallback strategies"""
# Perceive input
perception = self.safe_perceive("user", user_input)
if not perception["success"]:
return "I'm having trouble understanding that. Could you rephrase?"
# Try primary action
action = self.safe_act("primary_tool", {"query": user_input})
if not action["success"]:
# Fallback: Try alternative action
action = self.safe_act("fallback_tool", {"query": user_input})
if not action["success"]:
return "I tried to help, but encountered an error. Please try again."
return action["result"]
## Example usage
tools = {
"primary_tool": lambda x: f"Processed: {x['query']}",
"fallback_tool": lambda x: f"Alternative processing: {x['query']}"
}
agent = RobustAgent(tools)
result = agent.run_with_fallback("Help me with something")
print(result)Processed: Help me with something
This example shows defensive programming. The agent checks whether perceptions succeeded, whether tools are available, and has fallback strategies when primary actions fail.