Agent Safety › Summary

Agent safety is not a single feature but a layered system of complementary defenses. The key concepts from this chapter are:

• Action space constraints enforce the principle of least privilege, limiting what agents can do by design rather than relying on correct behavior at runtime. Action profiles and validators make these constraints explicit and enforceable.
• Sandboxing creates a technical barrier that prevents agents from accessing resources outside their designated environment, particularly important for agents with code execution capabilities. Container-based sandboxing provides good security at low overhead; gVisor and WASM provide stronger guarantees at higher cost.
• Monitoring and observability provide visibility into agent behavior in real time, supporting both rule-based alerting for known failure modes and statistical anomaly detection for unexpected patterns. The action log is the foundation of every other monitoring capability.
• Human-in-the-loop intervention through confirmation gates, graceful abort, and checkpoint recovery ensures that high-stakes actions receive appropriate human oversight and that failures are recoverable rather than catastrophic.
• Prompt injection defense through structural separation and sanitization, supported by content risk classification, protects agents from adversarial content embedded in the data they process. No single defense is complete; layering is essential.
• Safe design patterns including minimal footprint, checkpoint and recovery, and dual-layer architecture embed safety considerations into the agent's architecture from the start, rather than adding them as afterthoughts.
• Alignment verification through behavioral testing, output auditing, and ongoing operational review addresses the deeper question of whether the agent's behavior matches what was intended, beyond just preventing obvious harms.

The limitations, specification completeness, scalable oversight, adversarial resistance, and the open-world problem, remain active research challenges. The practical takeaway is to treat agent safety as defense in depth: no single mechanism is sufficient, but multiple overlapping layers significantly reduce the likelihood that any single failure causes serious harm. Start with narrow permissions and detailed logging. Use conservative confirmation requirements. Expand autonomy gradually, as operational experience builds confidence in the agent's behavior in the specific deployment context.