Michael Brenndoerfer | Data & AI, Private Equity, Technology

Michael Brenndoerfer

Ethical Guidelines and Human Oversight › Periodic Review and Updates

Michael BrenndoerferAugust 16, 20253 min read

AI Agent HandbookAll chapters

Part 13 of 15Safety and Governance - Keeping the Agent Aligned

Chapter 38Ethical Guidelines and Human Oversight › Periodic Review and Updates

Reading section 7 of 12 · Periodic Review and Updates · 3 minSection 7 of 12 · 3 min

Section 7 of 12 · 3 min read

Ethical Guidelines and Human Oversight › Periodic Review and UpdatesOne section · Sections in this chapter7 / 12 ↓

Sections in this chapter12

Ethical guidelines and oversight aren't set-it-and-forget-it. As your agent is used in the real world, you'll discover edge cases, user concerns, and new ethical challenges. You need a process for reviewing and updating your governance approach.

Establishing a Review ProcessLink Copied

For our personal assistant, here's a simple review process:

Weekly: Review flagged actions and audit logs

Look for patterns in what gets flagged
Check if the agent is refusing appropriate requests or allowing inappropriate ones
Adjust oversight thresholds if needed

Monthly: Review ethical guidelines

Have there been situations where the guidelines were unclear?
Are there new capabilities that need ethical guidance?
Have user needs or values changed?

Quarterly: Comprehensive governance review

Test the agent with challenging ethical scenarios
Review bias testing results
Update system prompts and oversight rules
Document changes and reasoning

Here's a simple tool for tracking governance issues:

In[6]:

Code

## Using Claude Sonnet 4.5 for governance tracking
import datetime


class GovernanceTracker:
    def __init__(self):
        self.issues = []
        self.reviews = []

    def log_issue(self, category, description, severity):
        """Log a governance issue for review"""
        self.issues.append(
            {
                "timestamp": datetime.datetime.now().isoformat(),
                "category": category,
                "description": description,
                "severity": severity,
                "status": "open",
            }
        )

    def conduct_review(self, review_type, findings, actions_taken):
        """Document a governance review"""
        self.reviews.append(
            {
                "timestamp": datetime.datetime.now().isoformat(),
                "type": review_type,
                "findings": findings,
                "actions_taken": actions_taken,
            }
        )

        # Close related issues
        for finding in findings:
            for issue in self.issues:
                if (
                    issue["status"] == "open"
                    and finding in issue["description"]
                ):
                    issue["status"] = "resolved"

    def get_open_issues(self, severity=None):
        """Get open governance issues"""
        issues = [i for i in self.issues if i["status"] == "open"]
        if severity:
            issues = [i for i in issues if i["severity"] == severity]
        return issues

    def generate_governance_report(self):
        """Generate a governance status report"""
        open_issues = self.get_open_issues()
        recent_reviews = sorted(
            self.reviews, key=lambda x: x["timestamp"], reverse=True
        )[:5]

        report = "GOVERNANCE STATUS REPORT\n\n"
        report += f"Open Issues: {len(open_issues)}\n"
        report += f"Total Reviews: {len(self.reviews)}\n\n"

        if open_issues:
            report += "OPEN ISSUES:\n"
            for issue in open_issues:
                report += f"  [{issue['severity']}] {issue['description']}\n"

        if recent_reviews:
            report += "\nRECENT REVIEWS:\n"
            for review in recent_reviews:
                report += f"  {review['type']}: {review['findings']}\n"

        return report


## Example usage
tracker = GovernanceTracker()

## Log issues as they arise
tracker.log_issue(
    category="bias",
    description="Agent made assumption about user's role based on name",
    severity="medium",
)

tracker.log_issue(
    category="autonomy",
    description="Agent canceled meeting without asking",
    severity="high",
)

## Conduct periodic review
tracker.conduct_review(
    review_type="weekly",
    findings=["Agent canceled meeting without asking"],
    actions_taken=[
        "Updated system prompt to require confirmation for cancellations"
    ],
)

## Generate report
print(tracker.generate_governance_report())

Out[6]:

Console

GOVERNANCE STATUS REPORT

Open Issues: 1
Total Reviews: 1

OPEN ISSUES:
  [medium] Agent made assumption about user's role based on name

RECENT REVIEWS:
  weekly: ['Agent canceled meeting without asking']