Ethical Guidelines and Human Oversight › Governance for Low-Stakes vs. High-Stakes Agents
AI Agent HandbookAll chapters Ethical Guidelines and Human Oversight › Governance for Low-Stakes vs. High-Stakes AgentsOne section · Sections in this chapter8 / 12 ↓
The governance needs for our personal assistant (relatively low-stakes) are different from an agent making medical recommendations or financial decisions (high-stakes). Let's contrast the two:
Low-Stakes Agent (Personal Assistant)
Ethical guidelines: Encoded in system prompts, relatively informal
Human oversight: Mostly human-out-of-loop with audit logging, human-in-loop for a few high-risk actions
Review process: Periodic self-review by the developer/user
Documentation: Simple logs and issue tracking
Accountability: Developer is accountable to themselves or small user base
High-Stakes Agent (Medical/Financial)
Ethical guidelines: Formal policy documents, reviewed by ethics committees, encoded in multiple layers
Human oversight: Extensive human-in-loop for most decisions, formal approval processes
Review process: Regular audits by external reviewers, compliance checks
Documentation: Comprehensive audit trails, decision justifications, regulatory reporting
Accountability: Organization is accountable to regulators, patients, customers, and public
For our personal assistant, we can keep governance relatively lightweight:
## Using Claude Sonnet 4.5 for lightweight governance
class PersonalAssistantGovernance:
def __init__(self):
self.ethical_guidelines = """
Core principles:
1. Respect user autonomy (ask before major decisions)
2. Protect privacy (don't share personal info)
3. Be fair (no bias or discrimination)
4. Be transparent (explain actions and reasoning)
5. Consider impact (think about effects on others)
"""
self.oversight_config = {
"send_email_external": "human_in_loop",
"cancel_meeting": "human_in_loop",
"send_email_internal": "human_on_loop",
"create_reminder": "human_out_of_loop",
"answer_question": "human_out_of_loop",
}
self.audit_log = []
def get_oversight_level(self, action_type):
"""Get required oversight for an action"""
return self.oversight_config.get(action_type, "human_on_loop")
def log_action(self, action_type, details, outcome):
"""Log an action for audit"""
self.audit_log.append(
{
"timestamp": datetime.datetime.now().isoformat(),
"action": action_type,
"details": details,
"outcome": outcome,
}
)
def weekly_review(self):
"""Simple weekly governance review"""
print("WEEKLY GOVERNANCE REVIEW\n")
print(f"Actions this week: {len(self.audit_log)}")
# Check for any concerning patterns
action_types = [a["action"] for a in self.audit_log]
from collections import Counter
counts = Counter(action_types)
print("\nAction breakdown:")
for action, count in counts.most_common():
print(f" {action}: {count}")
print("\nReview questions:")
print("- Were any actions inappropriate?")
print("- Should any oversight levels be adjusted?")
print("- Are ethical guidelines being followed?")
print("- Any new ethical concerns to address?")
## Example usage
governance = PersonalAssistantGovernance()
## Check oversight requirements
print(governance.get_oversight_level("send_email_external")) # human_in_loop
## Log actions
governance.log_action("answer_question", "Answered weather query", "success")
governance.log_action("create_reminder", "Set reminder for meeting", "success")
## Periodic review
governance.weekly_review()human_in_loop WEEKLY GOVERNANCE REVIEW Actions this week: 2 Action breakdown: answer_question: 1 create_reminder: 1 Review questions: - Were any actions inappropriate? - Should any oversight levels be adjusted? - Are ethical guidelines being followed? - Any new ethical concerns to address?
This lightweight approach is appropriate for a personal assistant. It provides structure without being burdensome.