Michael Brenndoerfer | Data & AI, Private Equity, Technology

Michael Brenndoerfer

Perception and Action › How Actions Change the Environment

Michael BrenndoerferJuly 15, 20252 min read

AI Agent HandbookAll chapters

Part 8 of 15Environment and Interaction - The Agent's World

Chapter 22Perception and Action › How Actions Change the Environment

Reading section 5 of 10 · How Actions Change the Environment · 2 minSection 5 of 10 · 2 min

Section 5 of 10 · 2 min read

Perception and Action › How Actions Change the EnvironmentOne section · Sections in this chapter5 / 10 ↓

Sections in this chapter10

Let's be specific about what "changing the environment" means for our assistant. Every action has consequences:

Text responses change the conversation state: When the agent replies, the conversation history grows. This changes what the agent will perceive in future cycles.

Tool calls change external systems: Sending an email, creating a calendar event, or updating a database all modify the external environment. These changes persist even after the agent stops running.

Memory updates change the agent's knowledge: When the agent saves information, it changes its own internal environment. Future perceptions will include this stored knowledge.

Failed actions create new perceptions: If a tool call fails, the agent perceives an error. This might trigger a different action, like trying an alternative approach or asking the user for help.

Example: Action Consequences (GPT-5)Link Copied

Let's see how one action creates ripple effects:

In[6]:

Code

import openai

## Using GPT-5 for this straightforward example
client = openai.OpenAI(api_key=os.getenv("OPENAI_API_KEY"))

def demonstrate_action_consequences():
    """Show how actions change what the agent perceives next"""
    
    # Initial state
    environment = {
        "calendar": [],
        "conversation": []
    }
    
    # Action 1: Agent adds event to calendar
    event = {"title": "Team meeting", "time": "2pm"}
    environment["calendar"].append(event)
    environment["conversation"].append({
        "role": "agent",
        "content": "I've scheduled the team meeting for 2pm."
    })
    
    print("After Action 1:")
    print(f"Calendar: {environment['calendar']}")
    
    # Action 2: User asks about schedule
    # Agent now perceives the event it created
    environment["conversation"].append({
        "role": "user",
        "content": "What's on my calendar?"
    })
    
    # Agent perceives its own previous action
    events = environment["calendar"]
    response = f"You have {len(events)} event(s): {events[0]['title']} at {events[0]['time']}"
    
    print(f"\nAgent perceives its own action: {response}")
    
    return environment

result = demonstrate_action_consequences()

Out[6]:

Console

After Action 1:
Calendar: [{'title': 'Team meeting', 'time': '2pm'}]

Agent perceives its own action: You have 1 event(s): Team meeting at 2pm

The agent's first action (scheduling the meeting) changed the environment. When the agent later perceives the calendar, it sees the result of its own previous action. This is how the perception-action cycle creates continuity.