Agent Safety › Action Space Constraints
Language AI HandbookAll chapters Agent Safety › Action Space ConstraintsOne section · Sections in this chapter3 / 12 ↓
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
The most direct way to limit agent harm is to limit what actions the agent can take. Every agent operates over an action space: the set of operations it can execute. Reducing that space to only what is necessary for the task is the first line of defense.
The Principle of Least Privilege
This principle, borrowed from computer security, states that a system should be granted only the minimum permissions necessary to perform its intended function. Applied to agents, it means that an agent given a task involving web search and text summarization should not have access to a code execution environment, a file system, or payment APIs. If you do not give the agent the tools to cause a certain category of harm, it cannot cause it.
The elegance of least privilege is that it converts a runtime correctness problem into a static configuration problem. Instead of asking "will this agent behave correctly when it has access to destructive tools?", you ask "does this agent need destructive tools at all?" For most tasks, the answer is no. An agent that summarizes documents does not need to send emails. An agent that answers questions about a knowledge base does not need to write files. Every tool you remove from the agent's action space is a category of harm you have eliminated entirely.
Implementing least privilege requires being explicit about which tools an agent can use, not just which tools it has access to. Many frameworks allow you to define a tool registry and then pass a subset of that registry to a given agent instance. The agent only sees the tools it needs, which limits the blast radius of any mistake or adversarial input.
Consider how this applies in practice. An agent tasked with answering questions about a company's internal knowledge base needs read access to documents. It does not need write access, email-sending capabilities, or the ability to execute shell commands. Granting those capabilities "just in case" is precisely the kind of over-provisioning that turns small errors into large incidents. The phrase "just in case" is a signal that you are about to violate least privilege.
Static vs. Dynamic Action Constraints
Constraints can be applied statically at agent initialization or dynamically during execution. Static constraints are simpler and more predictable: you define the tool set at startup and it does not change. Dynamic constraints adjust based on context, such as allowing more actions when operating in a staging environment and fewer in production, or restricting certain actions based on the sensitivity of the data being processed.
Each approach has tradeoffs. Static constraints are easier to reason about and audit because the permission set is constant and inspectable at configuration time. Dynamic constraints are more flexible and can adapt to changing conditions, but they introduce the question of what logic governs when permissions expand or contract. That logic itself becomes a safety-critical component that must be carefully designed and tested.
A useful pattern is to define action profiles: named sets of permissions appropriate for different operational contexts.
from dataclasses import dataclass, field
from enum import Enum
from typing import Set
class ActionProfile(Enum):
READ_ONLY = "read_only"
STANDARD = "standard"
ELEVATED = "elevated"
@dataclass
class AgentPermissions:
profile: ActionProfile
allowed_actions: Set[str] = field(default_factory=set)
denied_actions: Set[str] = field(default_factory=set)
requires_confirmation: Set[str] = field(default_factory=set)
def build_permissions(profile: ActionProfile) -> AgentPermissions:
if profile == ActionProfile.READ_ONLY:
return AgentPermissions(
profile=profile,
allowed_actions={"read_file", "search_web", "query_database"},
denied_actions={
"write_file",
"delete_file",
"send_email",
"execute_code",
},
requires_confirmation=set(),
)
elif profile == ActionProfile.STANDARD:
return AgentPermissions(
profile=profile,
allowed_actions={
"read_file",
"write_file",
"search_web",
"query_database",
},
denied_actions={"delete_file", "send_email", "execute_code"},
requires_confirmation={"write_file"},
)
elif profile == ActionProfile.ELEVATED:
return AgentPermissions(
profile=profile,
allowed_actions={
"read_file",
"write_file",
"delete_file",
"search_web",
"query_database",
"send_email",
"execute_code",
},
denied_actions=set(),
requires_confirmation={"delete_file", "send_email", "execute_code"},
)Action profiles:
READ_ONLY - Allowed: 3 actions, Denied: 4 actions
STANDARD - Allowed: 4 actions, Denied: 3 actions, Requires confirmation: {'write_file'}
ELEVATED - Allowed: 7 actions, Denied: 0 actions, Requires confirmation: {'delete_file', 'send_email', 'execute_code'}The read-only profile provides a safe default for agents that primarily consume information. The standard profile allows limited write operations but gates them behind confirmation. The elevated profile grants broad capabilities but requires explicit human approval for the highest-risk actions.
Notice how the three profiles form a natural hierarchy. READ_ONLY is appropriate for agents used in sensitive contexts where even write operations could cause problems, such as an agent querying production databases or reading confidential documents. STANDARD is appropriate for most development and operational tasks where write access is needed but destructive operations are not. ELEVATED is for exceptional cases where an agent needs to take actions that cannot easily be reversed, and the human-confirmation requirement ensures those actions are deliberate.
The heatmap below makes the permission structure concrete: each cell shows whether a given action is allowed (green), denied (red), or allowed with confirmation (yellow) for each profile.
Action Validators
Even with a constrained action space, individual action invocations need validation before execution. An action validator inspects each tool call before it reaches the actual tool, checking whether it violates any safety rules. The permission profile tells you what categories of actions are allowed; the validator inspects the specific arguments of each call to catch violations that profile-level restrictions cannot detect.
Consider the distinction carefully. A profile might allow read_file, but that does not mean every possible file path argument is safe. An agent instructed to read files might receive (through a prompt injection) a path like ../../etc/passwd that traverses outside the intended working directory. The profile says "reading files is allowed"; the validator says "but not that file". These two checks are complementary: neither is sufficient on its own.
import re
from typing import Any, Dict
class ActionValidationError(Exception):
pass
class ActionValidator:
def __init__(self, permissions: AgentPermissions):
self.permissions = permissions
self._dangerous_path_patterns = [
r"^\s*/etc/",
r"^\s*/sys/",
r"^\s*/proc/",
r"\.\./", # path traversal
r"~/", # home directory escape
]
def validate(self, action_name: str, action_args: Dict[str, Any]) -> None:
"""Raises ActionValidationError if action is not allowed."""
# Check against deny list
if action_name in self.permissions.denied_actions:
raise ActionValidationError(
f"Action '{action_name}' is in the denied actions list "
f"for profile {self.permissions.profile.value}"
)
# Check against allow list (if defined)
if (
self.permissions.allowed_actions
and action_name not in self.permissions.allowed_actions
):
raise ActionValidationError(
f"Action '{action_name}' is not in the allowed actions list"
)
# File path safety checks
if action_name in {"read_file", "write_file", "delete_file"}:
path = action_args.get("path", "")
for pattern in self._dangerous_path_patterns:
if re.search(pattern, path):
raise ActionValidationError(
f"File path '{path}' matches a restricted pattern"
)
def requires_confirmation(self, action_name: str) -> bool:
return action_name in self.permissions.requires_confirmationValidation results:
read_file({'path': '/home/user/report.txt'}) [should pass] -> PASSED
delete_file({}) [should fail (denied)] -> BLOCKED: Action 'delete_file' is in the denied actions list for profile standard
write_file({'path': '../../etc/passwd'}) [should fail (path traversal)] -> BLOCKED: File path '../../etc/passwd' matches a restricted pattern
send_email({}) [should fail (denied)] -> BLOCKED: Action 'send_email' is in the denied actions list for profile standardThe validator enforces permission boundaries at the point of execution. This means even if the agent's reasoning produces a plan that includes a disallowed action, the action will be caught before any harm occurs. The validator acts as the last line of defense before the action reaches the actual tool implementation.
A well-designed validation layer can also log every attempted violation. This creates an audit trail of what the agent tried to do, not just what it succeeded in doing. Attempted violations are often more informative than successful actions because they reveal cases where the agent's behavior diverged from its intended task.