Self-Models, Body Schemas, and Affordances › Modeling Other Agents and Theory of Mind

So far, "the agent" has been singular. Real environments contain other agents: people in a kitchen, cars at an intersection, collaborative robots on a line. They are not passive matter. They may act toward goals and react to us, with beliefs or intentions that no sensor reads directly. When those hidden variables materially change behavior, representing them can improve prediction and planning. Simpler reactive motion models may still be adequate for short horizons or weakly coupled encounters; the need for mental-state structure depends on the interaction.

The error is structured rather than merely independent noise, but it does not have a universal sign. A constant-velocity pedestrian model is wrong when the pedestrian notices the robot and adjusts; whether it over- or under-predicts progress depends on whether the person slows, accelerates, or changes direction. A robot that assumes people will yield can also behave in ways that force them to yield, degrading safety and cooperation. In interactions where such responses matter, goal- or belief-conditioned variables offer information that a momentum-only model omits.

The phrase "theory of" in the name is apt: the observer holds an inferential model, not a direct record of another mind. Predictive performance is one important criterion for that model. Causal and representational validity matter as well when the goal is explanation, intervention, or transfer rather than trajectory forecasting alone. This is why the evaluation difficulties discussed in Limitations cannot be reduced to a single prediction score.

The modeling move is one you have already made twice in this book: treat another agent's mental state as a latent variable in a partially observable model, and infer it from behavior. This resembles inferring a hidden state in Bayesian Filtering and Belief States or a hidden physical parameter from motion. What changes is the interpretation and transition model: the latent may denote a goal or belief, and the observer adds a behavioral or policy model alongside the physical dynamics. That policy may be habitual, boundedly rational, or task-specific rather than perfectly rational. In the framework of Bayesian Filtering and Belief States, the observer's belief now factorizes as

$$b_t(z_{1:t}, g_{1:t}) \propto \; p(o_{1:t} \mid z_{1:t}, g_{1:t})\; p(z_{1:t}, g_{1:t}),$$

where:

• $b_t(z_{1:t}, g_{1:t})$: the observer's belief, a posterior distribution over the closed-loop state trajectory $z_{1:t}$ and the other agents' latent mental-state trajectory $g_{1:t}$
• $o_{1:t}$: the observations accumulated from time $1$ through $t$
• $z_{1:t}$: the closed-loop state trajectory through time $t$
• $g_{1:t}$: the trajectory of the other agents' latent mental states (goals, beliefs, intentions); $g_t$ collects their values at time $t$
• $p(o_{1:t} \mid z_{1:t}, g_{1:t})$: the likelihood of the observed history under a given state and mental-state trajectory
• $p(z_{1:t}, g_{1:t})$: the joint prior over the closed-loop state trajectory and the mental-state trajectory, encoding the initial distribution and the transition dynamics

This is Bayes' rule over trajectories: the posterior weight on a hypothesis about the world and the other agents' minds is proportional to how well that hypothesis explains the observed history, scaled by its prior probability. Note what is being conditioned on and what is being inferred: the observer never sees $g_t$ directly, only its behavioral shadows, the actions the other agent takes. The whole apparatus of theory of mind lives inside the likelihood term, which encodes the observer's model of how minds produce actions. If that model says agents act efficiently toward their goals, then observed efficient motion is evidence for the goal it serves; if the model says agents avoid what they believe to be dangerous, then observed avoidance is evidence about their beliefs. Change the behavioral model and you change which mental states the same behavior supports, which is why the behavioral model, not the Bayesian update, is where most of the modeling content resides.

The transition model must then close the loop: the other agent's actions are generated by their policy, which depends on their latent state, so the full transition becomes

$$\begin{aligned} a^j_t &\sim \pi^j(\cdot \mid b^j_t, g_t), \\ z_{t+1} &= F(z_t, a^{\text{self}}_t, a^j_t) + w_t, \end{aligned}$$

where:

• $a^j_t$: the other agent's action at time $t$
• $\pi^j(\cdot \mid b^j_t, g_t)$: agent $j$'s policy, a distribution over actions conditioned on its belief $b^j_t$ and its latent mental state $g_t$
• $b^j_t$: agent $j$'s own belief at time $t$, its posterior over the closed-loop state and the mental states it attributes to others
• $a^{\text{self}}_t$: the self-agent's own action at time $t$
• $w_t$: the process noise in the closed-loop dynamics

This is the formal heart of theory of mind in a world model: the environment's dynamics contain a term that is itself a policy conditioned on a hidden belief. Note the immediate consequence: from the self-agent's perspective, the environment is nonstationary. The other agent's policy changes as their beliefs change, often because of the self-agent's actions. This is the same nonstationarity that forced multi-agent frameworks like I-POMDPs (interactive POMDPs) to recursively nest belief representations.

It is worth tracing exactly how the self-agent's actions leak into the other agent's policy, because the pathway is indirect and easy to lose. The self-agent acts; the action changes the world; the other agent observes the changed world; the observation shifts their belief $b^j_t$; the shifted belief changes the distribution over their next action; the action enters the shared transition $F$. The self-agent's influence on the other's mind therefore travels through the environment, with a lag, and the other agent's response travels back the same way. This two-hop causal path is what makes interaction qualitatively different from mere coexistence in a shared space:

The difference between a physics-only predictor and a response-aware predictor can be illustrated with a synthetic corridor simulation. The simulated pedestrian slows when the robot approaches. The constant-velocity baseline ignores that response, while the second predictor uses a deliberately slightly misspecified yielding rule based only on the robot-distance sequence available at prediction time.

In the chapter's hierarchy, two level-0 agents may react only after their motion models register a conflict, while level-1 agents can anticipate some responses from inferred goals. The resulting nonstationarity is not arbitrary drift: it comes from another decision process responding to observations. A model that tracks a useful hypothesis about that process can predict part of the change instead of treating it all as noise.

For this chapter, it helps to use a local hierarchy for how deeply the model represents other minds. Other level-$k$ and cognitive-hierarchy conventions define the levels differently:

• Level 0: model other agents as moving objects with dynamics. This can be adequate for short horizons and weak coupling, but it may fail when safe behavior depends on predicting how the other agent will respond.
• Level 1: attribute goals and predict actions from goals, as in inverse planning: an agent's action is likely if it efficiently achieves some goal. Our worked example below is exactly level-1 goal inference.
• Level 2 and beyond: attribute beliefs about beliefs. "The pedestrian thinks I see them, so they will cross, so I should slow down." Formally, the other agent's belief $b^j$ includes a model of me, so my model of them must include their model of me. Recursive level-$k$ reasoning grounds game-theoretic solution concepts, and interactive POMDPs make the recursion explicit.

Each added level pays for representational depth with computation and new opportunities for error. The chapter's level 0 is weakest when behavior responds to the focal agent, while level 1 can represent goal-directed responses but not coordination driven by beliefs about what others know. Level 2 adds that higher-order structure and immediately raises the question of where to stop. Shallow recursive models are computationally practical and often explain behavior in experimental games, but useful depth is task-dependent; deeper reasoning is not confined to adversarial or high-stakes settings.

The recursion cannot go infinite, so practical agents truncate at some level and may combine shallow deliberate reasoning with fast reactive models. The reactive layer can handle routine, weakly coupled motion, while deeper modeling is reserved for interactions where another agent's response changes the decision. An autonomous-driving stack might therefore use constant-velocity prediction for distant cars, goal inference for cars at an intersection, and belief attribution for a pedestrian whose decision depends on whether they have been seen. This is an allocation decision whose adequacy must be tested against the scenarios the vehicle faces.

Three concrete payoffs justify the complexity:

• Prediction quality in interaction. In multi-agent scenes, the marginal distribution of others' motion depends on the self-agent's plan. A model that conditions others' behavior on the self-agent's intended actions makes fundamentally different (and better) predictions than one that does not.
• Communication and cooperation. Handing someone a cup requires modeling what they know about the handover and when they will close their hand. That is a belief-tracking problem before it is a motor problem.
• Safety. Many failures of deployed agents come from under-modeling people: treating a human as a static obstacle produces plans that are technically feasible and socially disastrous. Over-modeling has a mirror-image failure, which we return to in Limitations.

The first payoff deserves emphasis. When another agent materially responds to the focal agent, the environment's transition distribution depends on the focal agent's behavior. Omitting that dependence then structurally misspecifies the interaction model rather than merely adding observation noise. Weakly coupled scenes and short horizons may still admit a useful simpler approximation, so the modeling depth should follow the sensitivity of the decision to the other agent's response.

There is also an introspective payoff. A self-model that includes "what others can observe about me" supports epistemic behaviors like signaling, feinting, and privacy, and it is the seed of the social self. Notice that this closes a loop with the beginning of the chapter: the agent-environment boundary, introduced as a private modeling choice, has a public counterpart, since other agents maintain their own models of what lies inside ours. An agent that represents the fact that it is in someone else's environment model has crossed from modeling the world to modeling itself as part of the world. We will revisit agent models at foundation scale in Part IX: Foundation and World-Action Models. Some foundation models display theory-of-mind-like behavior on particular benchmarks, but performance is uneven, and neither the underlying mechanism nor the construct validity of those benchmarks is settled.