Self-Models, Body Schemas, and Affordances

Imagine a warehouse robot arm asked to pick up a mug from a cluttered shelf. To solve this, the arm's world model must represent far more than the mug. It must know where its own joints are right now, how its end effector moves when a command is issued, which shelf locations it can physically reach from its current posture, and whether the mug is graspable given its gripper. In deployments around people, it may also benefit from representing goals or beliefs when those hidden variables improve predictions of how a nearby person will respond. These quantities concern the relationship between a specific agent and a specific world, not only the mug's intrinsic physics.

Notice how different these representations are from the object models of earlier chapters. A model of how a mug slides, tips, or breaks can transfer across agents; a reachability map is tied to a particular embodiment and state. A mug's mass does not depend on who holds it, while grasp success depends on the gripper, controller, and posture. A robot with an environment model but no representation of its own action interface is like a chess engine that knows the rules but not which pieces it controls: simulation alone does not tell it which interventions are available.

This chapter is about the agent-shaped part of a world model. Previous chapters built up representations of the environment: sufficient state, latent variables, temporal memory, objects, and relations. Here we complete the picture by turning the modeling lens back onto the agent itself. We cover four connected ideas:

• Agent-environment boundaries: the modeling choice of what counts as "me" versus "the world," and why the boundary is a design decision with real consequences.
• Body schemas and kinematic state: the internal model of the agent's own geometry and motion, including proprioception and learned self-models.
• Capabilities, constraints, and affordances: compact summaries of what is possible for this agent in this state, which connect prediction to decision-making.
• Other agents and theory of mind: models of other intentional systems, whose latent mental states make the world nonstationary from a single-agent viewpoint.

These four ideas form a natural progression, and it is worth seeing the progression explicitly before we begin. First the agent must decide where it ends and the world begins, because every subsequent representation is relative to that choice. Then it must model the region it claimed as itself: its geometry, its motion, how commands become movements. Then it must compress that model into statements of possibility, because planners cannot afford to reason from raw kinematics at every step. Finally, it must extend the same machinery to minds, because other agents are the part of the environment whose behavior is best explained not by physics but by goals.

The through-line is a single question: what does a world model need to represent about the agent so that its predictions are useful for the agent's own decisions? Passive prediction quality alone is not enough. A model that predicts the next camera frame but cannot condition forecasts on candidate actions cannot plan with those forecasts. An explicit factorization of action-controllable latents is not logically required, but it can make planning more efficient and the model easier to diagnose. This is one instance of a distinction this book presses repeatedly: predictive fidelity, representation quality, uncertainty quality, and decision usefulness are separate achievements. Self-models make that distinction concrete, and affordances are a representation built explicitly for decision use at some cost to generality.

In the controlled dynamical systems we studied in Part II: Inference, Decisions, and Control, we wrote the world as a transition model $z_{t+1} = F(z_t, a_t) + w_t$, where $w_t$ is process noise, and treated $a_t$ as an input channel distinct from everything else. But which physical variables belong inside $z_t$, and which are left outside as "environment"? For a robot arm gripping a mug, the joint angles belong to the agent. The mug's pose belongs to the environment. What about the mug's pose relative to the gripper once contact is made? What about a tool the arm is holding? The placement of those variables is a modeling choice with downstream consequences.

It matters because the two sides of the boundary are usually treated differently downstream. Variables on the self side often have stronger action coupling and more direct proprioceptive readout, so the agent may have lower-latency, higher-confidence estimates of them. Variables on the environment side rely more heavily on prediction and inference, with greater uncertainty and delay. These are tendencies rather than definitions: internal actuator states can be hidden, while an external object can be observed precisely. If a variable that behaves like part of the body is filed under "environment," the agent may underuse reliable action and sensor structure. If a variable that behaves like the world is filed under "self," the agent may become overconfident about something it cannot command. The Kalman filters and controllers of Part II: Inference, Decisions, and Control assumed this partition was given to them; learning agents must discover or design it, and the choice shapes everything downstream, from which latent dimensions receive action inputs to which parts of the model transfer to a new task.

Three properties guide where to draw the line:

To make the criteria concrete, score a set of representative variables on the first two properties: how directly a command moves them, and how directly a sensor reads them. Variables near the top right are natural members of $z^{\text{self}}$; variables near the bottom left belong to the environment; and the off-diagonal quadrants are exactly the ambiguous cases the text describes.

• Direct controllability: variables in $z^{\text{self}}$ respond to actions through short, reliable causal paths. Commands change joint torques within milliseconds; they change the mug's location only through contact dynamics.
• Direct observability: dedicated sensors such as joint encoders, IMUs, muscle spindles, and battery monitors provide measurements closely tied to self-state, often with less scene interpretation than cameras or lidar. They can still require filtering and state estimation; an IMU, for example, does not directly report pose.
• Transportability across tasks: the self-model is the part of the world model that persists when you move the robot to a new kitchen. The mug model does not. Factoring the self out of the environment model is therefore a powerful form of transfer: you can relearn the kitchen without relearning your arm.

Each of these criteria can fail independently. A variable can be controllable but hard to observe: joint torques respond to commands, but some robots lack direct joint-torque sensing and must estimate them. A variable can be observable but not controllable: a fixed camera sees the room but cannot move it. A variable can be both controllable and observable yet fail to transfer: a gripper's learned friction model depends on the objects it has gripped. Transportability is valuable because the self-model is meant to be reused, but whether a variable belongs there still depends on the downstream task.

This connects directly to the state-abstraction machinery from Sufficient State and State Abstraction in this part. There, we asked which observation features a policy needs. Here we ask a complementary question: which latents are action-controllable, and which are merely observable? In a learned latent dynamics model, some latent dimensions will move sharply and predictably under actions while others respond only weakly or not at all. Identifying a controllable subspace is one useful step toward discovering the body. In classical robotics the partition is handed to you: the manufacturer tells you which encoders are on the arm. In a learned world model trained purely on observation-action sequences, the model instead has to infer which dimensions its actions steer. Short response latency, action-conditioned mutual information, and persistence across scenes are useful heuristics for proposing controllable latents, but none uniquely identifies self-state. Recovering a defensible self/nonself split therefore requires intervention and transfer tests, not a single statistical signature.

The boundary is also not fixed. When a person uses a rake, their effective reaching range changes; neuropsychologists observe that peripersonal space (the region the brain treats as "within reach") expands to include the tool. A good computational analogue: after a few minutes of driving, a driver's self-model extends to the car's corners, and experienced drivers know "where the hood is" without seeing it. We will return to tool incorporation when we discuss learned body schemas. The plasticity of the boundary is not a biological curiosity; it is a design resource. A robot that can dynamically reclassify a grasped object as part of itself inherits a compact, already-calibrated kinematic model for it, rather than treating every object-hand interaction as a fresh contact-dynamics problem. The cost is symmetric: the boundary must also contract when the object is dropped, and a stale extension is worse than no extension, because the agent will confidently predict motion (its own, extended) that does not occur.

Finally, note a subtlety about actions themselves. In the POMDP formalism from Markov and Partially Observable Decision Processes, the action is an input, not part of the state. But real actuators have state: motor temperature, battery charge, worn gears, a gripper that is partially closed. A complete self-model includes these actuator states inside $z^{\text{self}}$, because they shape how commands translate into forces. Ignoring them is a classic source of model error that worsens over a long episode. The failure pattern is characteristic and worth recognizing: the model is accurate early in the episode, when actuator states are near their nominal values, and drifts later as batteries deplete and motors heat. An agent that evaluates its model only on short rollouts will never see the problem; an agent that tracks its own actuator states can predict the onset of the discrepancy, which is exactly the kind of self-knowledge that separates a system that fails gracefully from one that fails inexplicably.

Why can a cat pounce onto a narrow ledge without measuring anything, and why can you touch your nose with your eyes closed? Both rely on an internal model of the body's geometry and its current configuration. The cat does not compute the ledge height in world coordinates and compare it against a table of jump distances; it consults an integrated sense of where its body is and what it can do from here. You do not locate your nose visually; you consult a body-centered map that has been maintained continuously since long before you could read. We use body schema for the functional, and typically learned, model of the agent's own body and its relation to actions.

It is worth being precise about what "model" means here, because the body schema is not one function but a cluster of them, tied together by a shared subject. There is the command-to-configuration map (what does this motor command do to my joints?), the configuration-to-sensation map (what will I feel if my joints move like this?), and the configuration-to-geometry map (where is my hand relative to my shoulder?). Classical robotics often specifies these with separate analytic and calibrated models; a learned body schema may fit them jointly. What makes them a schema rather than a grab-bag of regressions is that they concern the same self-state and can be checked for cross-model consistency.

Recall from our discussion of sufficient state in Part II: Inference, Decisions, and Control that a Markov state must contain everything relevant to future prediction. For an embodied agent, the body configuration $q_t$ (joint angles, base pose, gripper aperture) is unambiguously part of that state. Omitting it has immediate consequences: two observationally identical camera views can correspond to completely different futures depending on arm posture, because one posture can reach the target and the other cannot. If your latent state does not encode posture, your model cannot predict which actions will work, and no amount of visual fidelity fixes it.

This can be a concrete instance of the partial-observability diagnosis from Markov and Partially Observable Decision Processes. If posture is hidden and the available camera history does not disambiguate action-relevant futures, a camera-only agent is partially observable rather than merely noisy. A vision-only latent model may still score well on passive frame prediction when the scene dominates the image and hidden posture has little visual effect. The deficiency appears under control, where posture changes which actions are viable. This is another reason passive prediction scores do not by themselves establish closed-loop competence.

This is why proprioception is not a nice-to-have sensor. Joint encoders and other proprioceptive sensors often provide high-rate, low-latency measurements of the agent's own state, although their reliability and calibration still depend on the hardware and estimator. In Part III: Representing Agents and Worlds's opening chapter on observations and sensors, we classified modalities by what they measure. Proprioception primarily measures the self; exteroception primarily measures the world. A world model for an embodied agent typically needs both, fused into a single latent state.

The minimal kinematic self-model has two parts:

• Forward kinematics: the map $f$ from joint configuration to body geometry, for example the end-effector position $x = f(q)$.
• The Jacobian: the derivative $J(q) = \partial f / \partial q$, which linearizes the map and tells you how end-effector motion responds to small joint motions. Resolved-rate controllers and many local inverse-kinematics methods consume $J(q)$ directly; model-predictive controllers may instead use other derivatives or a full dynamics model.

Why does the Jacobian deserve attention alongside the map itself? Local inverse kinematics asks which small joint adjustment moves the hand toward a target. The Jacobian answers that differential question. For redundant or constrained arms, one practical iterative method repeatedly solves a linearized problem $J(q)\,\Delta q = \Delta x$, takes a controlled step, and re-linearizes. Resolved-rate controllers and many local inverse-kinematics methods use some version of this loop, which is why the learned-schema experiment checks a derivative instead of reporting only point predictions.

The Jacobian's role in control can be seen directly by running the resolved-rate loop $J(q)\,\Delta q = \Delta x$ from a starting posture toward a target. The first few steps need not reduce the residual monotonically because each solve uses a local linearization. Once the arm enters the target's neighborhood, the damped updates produce near-geometric decay until floating-point precision becomes the limiting scale.

Dynamics (masses, inertias, gravity, friction, and contact forces) sit on top of kinematics. A kinematic schema answers "where will my hand be if my joints move like this"; a dynamic schema answers "what torques achieve that motion, and how fast." Kinematics can be sufficient for geometric path generation in slow, position-controlled manipulation when a lower-level controller handles the dynamic effects. It is not a claim that those effects disappear. Heavy payloads, fast motions, and contact-rich tasks make dynamics important at the planning layer as well. The classical robotics literature on computed-torque control and operational-space formulations shows how the two layers compose; a learned agent faces the same layering choice, deciding which dynamic structure must be explicit for the task.

Classical robotics writes $f(q)$ in closed form from CAD models. This works when the robot matches its blueprint but requires recalibration when damage, deformation, wear, or an unmodeled payload changes the plant. Bongard, Zykov, and Lipson demonstrated an adaptive alternative in their 2006 Science paper on resilient machines: a four-legged robot used actuation-sensation data to select a self-model, detected disagreement after damage, and synthesized a compensating gait. The lesson generalizes beyond that platform: a self-model estimated from data can be revised when observations reveal mismatch.

The important distinction is not analytic versus learned, but monitored versus unmonitored and fixed versus updateable. Either an analytic or learned model can be checked through prediction residuals. A parameterized or learned model may be easier to revise when those residuals reveal damage or wear. The resilient robot follows that loop: disagreement grows, the agent detects it, and it searches for an updated self-model. This is closely related to the filtering logic from Part II: Inference, Decisions, and Control, where an unexpectedly large innovation can trigger diagnosis or re-estimation.

Learning a body schema gives the agent unusual influence over its calibration data: it can choose safe motor commands and observe their consequences. That does not mean full control of the distribution. Joint limits, contacts, safety constraints, actuator health, and sensor noise restrict which probes are possible and what they reveal. Even so, self-model data can often be gathered more actively than data about an uncontrolled environment. A well-designed embodied agent can exploit that advantage through periodic calibration actions and residual monitoring, while respecting the same coverage and safety constraints that govern any system-identification experiment.

The same machinery that models the arm can model a held tool. Kinematically, a gripper holding a rigid rod is a chain with an added geometry and attachment transform. The agent can learn an extended forward map $f_{\text{tool}}(q, \theta^{\text{kin}}_{\text{tool}})$, where $\theta^{\text{kin}}_{\text{tool}}$ contains quantities such as length, shape, and grasp transform. A dynamic extension would separately include mass, inertia, compliance, and friction. Grasping a tool can therefore involve estimating parameters and selecting the appropriate extension. Within a fixed parameterized tool family, a new tool may be handled as new parameter values rather than an entirely new model. We will see the same compositional idea again when we treat object-centric structure in Object-Centric and Relational Worlds and physical constraints in Part IV: Space, Physics, and Causality.

The parameter-estimation view also suggests a computational account of the brief calibration period when a person picks up a new tool. The agent can actively probe length, orientation, and response, then use the estimate $\hat{\theta}_{\text{tool}}$ in an extended schema. This account predicts characteristic errors, such as reaching short with an underestimated rake length, that should shrink as calibration samples accumulate. Human and primate studies do show task-dependent changes in body and peripersonal-space representations after tool use. Those observations are qualitatively consistent with calibration, but they do not establish that the nervous system performs the specific parameter-estimation procedure written here.

Knowing your body configuration is not yet knowing what you can do. The next layer of the self-model summarizes possibility: which actions are feasible, which outcomes are reachable, and which interactions the environment currently supports. These summaries are where self-models first pay off in the decision loop, because they let a planner prune huge portions of the action space before any expensive rollout.

The need for this layer comes from a scaling problem that raw self-models cannot solve. A body schema plus a world model is, in principle, complete: for any candidate action sequence, the agent could roll the pair forward and find out what happens. But that completeness is combinatorially expensive, and most of the expense is wasted on actions that were never viable. A planner that rolls out a grasp attempt in a region the arm cannot physically reach has not learned anything useful; it has paid full model-evaluation cost to rediscover a geometric fact it could have known in advance. Capability summaries are the cache that prevents this waste. They answer the coarse question "is this even possible?" cheaply, so that the expensive question "what exactly happens if I try?" is reserved for candidates that survive the first filter.

Let $\mathcal{C} \subseteq \mathbb{R}^n$ denote the set of body configurations the agent can occupy or command, where $n$ is the number of independent configuration variables (joint angles, base pose, gripper aperture). For a robot arm, $\mathcal{C}$ is carved out by several families of constraints:

• Joint limits: each joint angle lies in an interval.
• Actuation limits: torques and velocities stay within motor capabilities, which couple across joints.
• Self-collision: the body must not intersect itself, a nonconvex geometric constraint.
• Stability: legged and wheeled systems must keep their center of mass and contact forces within support regions.
• Task constraints: keep the cup upright, keep the end effector out of the sterile field.

These constraint families have different mathematical forms. Joint limits are box constraints on individual variables. Self-collision is a nonlinear, nonconvex geometric predicate involving many variables at once. Stability constraints include dynamics and change with the contact state. Task constraints are imposed from outside the agent and may change with each job. A feasible set assembled from such pieces is often nonconvex and can be disconnected when obstacles, contacts, or task constraints separate regions of configuration space. Exact characterization can be expensive, so capability maps commonly approximate the forward-kinematic image of the feasible set by analysis, optimization, or sampling. Their reliability depends on the guarantee provided by the chosen method.

The part of the world that the agent's body can realize is then $f(\mathcal{C})$, the forward-kinematic image of the feasible set. For a two-link arm this image is an annulus: the end effector can reach anything between radius $|L_1 - L_2|$ and radius $L_1 + L_2$ from the shoulder, where $L_1$ and $L_2$ are the two link lengths, and nothing inside that inner hole. This geometric fact, derived entirely from the self-model with no reference to any object, is the prototype of a capability. The inner hole deserves a moment of reflection, because it is the most counterintuitive part of the result and the most decision-relevant. A planner without the capability map, shown a target near the shoulder, would try to plan a reach there, fail, retry with different parameters, and fail again; each failure costs a full planning episode. The map converts all of those episodes into a single lookup. And note that the knowledge is purely about the agent: the annulus exists whether or not there is a mug anywhere in the scene.

The "mediated by state" clause is doing real work in that definition. Capabilities sound like fixed properties of an agent, the way a spec sheet lists a robot's payload and reach, but the useful notion is state-dependent: a folded arm cannot reach what an extended one can, a grasping gripper cannot pinch, a robot with a depleted battery has a smaller feasible torque set. Capabilities are therefore best understood as functions of the self-state, not constants, and the capability summaries that matter in practice are indexed by the same $z^{\text{self}}_t$ that the body schema models.

The ecological psychologist James J. Gibson introduced and developed the affordance concept in his 1966 work, then gave it its canonical mature treatment in The Ecological Approach to Visual Perception (1979). Standard physics vocabulary can describe a flat rigid surface at knee height, but it does not directly say whether a particular body can sit on it. Affordances express that relation between environment and agent. A bench's height in centimeters is an object property; "sit-on-able" is an action-relevant summary, and it depends on the sitter.

::: {.callout-note title="Affordance"} An affordance is a relation between an agent's capabilities and an environmental state that constrains which interactions are possible and what outcomes they may produce. In this chapter, we operationalize it as a skill-, outcome-, and evaluation-horizon-conditioned prediction. For an observation $o_t$, a skill $k$, an outcome class $e$, and a horizon $H$,

where:

• $\Phi_{k,H}$: the affordance function for skill $k$ and evaluation horizon $H$, returning the probability that executing $k$ from this situation yields outcome class $e$ within that horizon
• $e$: the outcome class whose success probability is being predicted
• $k$: the skill being executed
• $H$: the evaluation horizon within which success must occur
• $o_t$: the observation (or state) at time $t$
• $z^{\text{self}}_t$: the agent's self-state at time $t$ (its body configuration together with its actuator states)

An affordance holds when this probability exceeds a decision threshold.

The threshold clause is the hinge between graded prediction and discrete commitment. For a fixed skill and scene, the success probability varies with the self-state (here, reach extension); the planner's go/no-go boundary is where the curve crosses the threshold.

:::

Several features of this definition deserve attention.

• Affordances are conditioned on the self. The same visual scene affords different actions to a robot with a parallel-jaw gripper versus a suction gripper. This is exactly why affordances live in this chapter rather than in the environment-modeling chapters: the function $\Phi_{k,H}$ takes the self-state as an argument. Strip the $z^{\text{self}}_t$ argument out and the function silently changes meaning, from "what can this agent do here?" to "what can some agent do here?", and the resulting quantity is much less useful for the agent's own planning.
• Affordances are skill-relative. "Graspable" is not a property of the mug; it is a property of the mug under a particular grasp policy. If the policy improves, the affordance map changes. Sahin and colleagues formalized this agent-relativity in their 2007 treatment of affordances in robotics, and it remains the standard framing. This skill-relativity has a practical implication that is easy to miss: affordance estimates are only as good as the policies they were learned under, and reusing an affordance map after upgrading the skill is a form of stale-model error, exactly analogous to using a kinematic schema after the robot has been repaired.
• Affordances are decision-useful summaries. Here we connect to a theme this book returns to repeatedly: the distinction between open-loop prediction quality and closed-loop usefulness. A full world model can answer "what happens if I execute the grasp skill here?" by rolling out predictions. An affordance map answers it in one lookup, at the cost of discarding information irrelevant to that action-outcome pair. Affordances are, in the language of Sufficient State and State Abstraction, an abstraction of the world model tuned to a decision problem. They trade generality for cheaper evaluation; any gain in robustness has to be demonstrated under the relevant shift.

That last point deserves a fuller statement, because it is the chapter's clearest example of the four-way split among notions of model quality introduced at the outset. An affordance map answers fewer questions than the world model it summarizes, and it answers them coarsely through the lossy bottleneck of a success probability. What it can buy in exchange is speed and task alignment: the map is cheaper to evaluate and can be learned directly from success labels. A task-aligned summary may transfer better when it discards nuisance variation while retaining the causal factors that determine success, but low dimensionality does not make stability under distribution shift automatic. Systems can therefore maintain both representations: a fuller model for novel situations and detailed planning, and an affordance layer for fast filtering when its calibration has been validated.

The reachability map of a robot arm is the cleanest example. Rather than asking, for every candidate goal, "can some sequence of joint motions bring the end effector here?", the agent precomputes or learns a map $\mathcal{R}: \mathcal{W} \to \{0,1\}$ over the workspace $\mathcal{W}$. A planner may safely prune a goal only when the map supplies a conservative infeasibility certificate; an unsampled cell is not such a certificate. The same pattern scales up. Modern robotics includes visual affordance models for action categories such as grasp, lift, open, support, push, and pull. Some predict semantic action regions; others estimate robot-specific success from interaction data. In each case, the representation compresses a class of counterfactual rollouts into a per-location or per-action score whose meaning depends on the skill, embodiment, and evaluation horizon.

There are two common routes, and they are often combined:

• Self-supervised exploration. The agent executes its skills across states and records successes and failures, then fits $\Phi_{k,H}$ from its own interaction log. This inherits the strengths of self-model learning: control over the sampled situations and continual recalibration. Its weakness is coverage: affordances for rare configurations are poorly estimated, and exploration in the real world is expensive.
• Transfer from demonstration or priors. Affordance structure observed in human data, other robots, or simulation seeds $\Phi$ before the agent has explored. This is fast but imports someone else's embodiment, so the imported affordances must be re-grounded in the agent's actual capabilities, which is precisely what the self-model provides.

At foundation scale, observation-based pretraining can supply useful structure before a robot acts, but the imported predictions still need robot-specific grounding. Human video reflects human hands, reach, scale, and force. Applied directly to a different embodiment, its action predictions can be miscalibrated in systematic ways. Robot interaction data and the self-model supply the correction through the robot's reach envelope, gripper geometry, controller competence, and force limits. This is a useful division of labor, not a guaranteed mathematical factorization: prior data propose environment-conditioned action structure, while robot-specific trials determine whether that structure predicts success for this skill and body.

We will meet the exploration side again in the planning chapters of Part VII: Planning and Agency, where information gathering about affordances competes with task progress. Every probe is a step not spent directly on the task, while a task attempt becomes an affordance sample only if the agent records the outcome. Too much probing delays commitment; too little produces overconfident extrapolation from sparse evidence. The uncertainty-aware treatment of affordance estimates in Part IV: Space, Physics, and Causality begins to formalize that tradeoff.

So far, "the agent" has been singular. Real environments contain other agents: people in a kitchen, cars at an intersection, collaborative robots on a line. They are not passive matter. They may act toward goals and react to us, with beliefs or intentions that no sensor reads directly. When those hidden variables materially change behavior, representing them can improve prediction and planning. Simpler reactive motion models may still be adequate for short horizons or weakly coupled encounters; the need for mental-state structure depends on the interaction.

The error is structured rather than merely independent noise, but it does not have a universal sign. A constant-velocity pedestrian model is wrong when the pedestrian notices the robot and adjusts; whether it over- or under-predicts progress depends on whether the person slows, accelerates, or changes direction. A robot that assumes people will yield can also behave in ways that force them to yield, degrading safety and cooperation. In interactions where such responses matter, goal- or belief-conditioned variables offer information that a momentum-only model omits.

The phrase "theory of" in the name is apt: the observer holds an inferential model, not a direct record of another mind. Predictive performance is one important criterion for that model. Causal and representational validity matter as well when the goal is explanation, intervention, or transfer rather than trajectory forecasting alone. This is why the evaluation difficulties discussed in Limitations cannot be reduced to a single prediction score.

The modeling move is one you have already made twice in this book: treat another agent's mental state as a latent variable in a partially observable model, and infer it from behavior. This resembles inferring a hidden state in Bayesian Filtering and Belief States or a hidden physical parameter from motion. What changes is the interpretation and transition model: the latent may denote a goal or belief, and the observer adds a behavioral or policy model alongside the physical dynamics. That policy may be habitual, boundedly rational, or task-specific rather than perfectly rational. In the framework of Bayesian Filtering and Belief States, the observer's belief now factorizes as

where:

• $b_t(z_{1:t}, g_{1:t})$: the observer's belief, a posterior distribution over the closed-loop state trajectory $z_{1:t}$ and the other agents' latent mental-state trajectory $g_{1:t}$
• $o_{1:t}$: the observations accumulated from time $1$ through $t$
• $z_{1:t}$: the closed-loop state trajectory through time $t$
• $g_{1:t}$: the trajectory of the other agents' latent mental states (goals, beliefs, intentions); $g_t$ collects their values at time $t$
• $p(o_{1:t} \mid z_{1:t}, g_{1:t})$: the likelihood of the observed history under a given state and mental-state trajectory
• $p(z_{1:t}, g_{1:t})$: the joint prior over the closed-loop state trajectory and the mental-state trajectory, encoding the initial distribution and the transition dynamics

This is Bayes' rule over trajectories: the posterior weight on a hypothesis about the world and the other agents' minds is proportional to how well that hypothesis explains the observed history, scaled by its prior probability. Note what is being conditioned on and what is being inferred: the observer never sees $g_t$ directly, only its behavioral shadows, the actions the other agent takes. The whole apparatus of theory of mind lives inside the likelihood term, which encodes the observer's model of how minds produce actions. If that model says agents act efficiently toward their goals, then observed efficient motion is evidence for the goal it serves; if the model says agents avoid what they believe to be dangerous, then observed avoidance is evidence about their beliefs. Change the behavioral model and you change which mental states the same behavior supports, which is why the behavioral model, not the Bayesian update, is where most of the modeling content resides.

The transition model must then close the loop: the other agent's actions are generated by their policy, which depends on their latent state, so the full transition becomes

where:

• $a^j_t$: the other agent's action at time $t$
• $\pi^j(\cdot \mid b^j_t, g_t)$: agent $j$'s policy, a distribution over actions conditioned on its belief $b^j_t$ and its latent mental state $g_t$
• $b^j_t$: agent $j$'s own belief at time $t$, its posterior over the closed-loop state and the mental states it attributes to others
• $a^{\text{self}}_t$: the self-agent's own action at time $t$
• $w_t$: the process noise in the closed-loop dynamics

This is the formal heart of theory of mind in a world model: the environment's dynamics contain a term that is itself a policy conditioned on a hidden belief. Note the immediate consequence: from the self-agent's perspective, the environment is nonstationary. The other agent's policy changes as their beliefs change, often because of the self-agent's actions. This is the same nonstationarity that forced multi-agent frameworks like I-POMDPs (interactive POMDPs) to recursively nest belief representations.

It is worth tracing exactly how the self-agent's actions leak into the other agent's policy, because the pathway is indirect and easy to lose. The self-agent acts; the action changes the world; the other agent observes the changed world; the observation shifts their belief $b^j_t$; the shifted belief changes the distribution over their next action; the action enters the shared transition $F$. The self-agent's influence on the other's mind therefore travels through the environment, with a lag, and the other agent's response travels back the same way. This two-hop causal path is what makes interaction qualitatively different from mere coexistence in a shared space:

The difference between a physics-only predictor and a response-aware predictor can be illustrated with a synthetic corridor simulation. The simulated pedestrian slows when the robot approaches. The constant-velocity baseline ignores that response, while the second predictor uses a deliberately slightly misspecified yielding rule based only on the robot-distance sequence available at prediction time.

In the chapter's hierarchy, two level-0 agents may react only after their motion models register a conflict, while level-1 agents can anticipate some responses from inferred goals. The resulting nonstationarity is not arbitrary drift: it comes from another decision process responding to observations. A model that tracks a useful hypothesis about that process can predict part of the change instead of treating it all as noise.

For this chapter, it helps to use a local hierarchy for how deeply the model represents other minds. Other level-$k$ and cognitive-hierarchy conventions define the levels differently:

• Level 0: model other agents as moving objects with dynamics. This can be adequate for short horizons and weak coupling, but it may fail when safe behavior depends on predicting how the other agent will respond.
• Level 1: attribute goals and predict actions from goals, as in inverse planning: an agent's action is likely if it efficiently achieves some goal. Our worked example below is exactly level-1 goal inference.
• Level 2 and beyond: attribute beliefs about beliefs. "The pedestrian thinks I see them, so they will cross, so I should slow down." Formally, the other agent's belief $b^j$ includes a model of me, so my model of them must include their model of me. Recursive level-$k$ reasoning grounds game-theoretic solution concepts, and interactive POMDPs make the recursion explicit.

Each added level pays for representational depth with computation and new opportunities for error. The chapter's level 0 is weakest when behavior responds to the focal agent, while level 1 can represent goal-directed responses but not coordination driven by beliefs about what others know. Level 2 adds that higher-order structure and immediately raises the question of where to stop. Shallow recursive models are computationally practical and often explain behavior in experimental games, but useful depth is task-dependent; deeper reasoning is not confined to adversarial or high-stakes settings.

The recursion cannot go infinite, so practical agents truncate at some level and may combine shallow deliberate reasoning with fast reactive models. The reactive layer can handle routine, weakly coupled motion, while deeper modeling is reserved for interactions where another agent's response changes the decision. An autonomous-driving stack might therefore use constant-velocity prediction for distant cars, goal inference for cars at an intersection, and belief attribution for a pedestrian whose decision depends on whether they have been seen. This is an allocation decision whose adequacy must be tested against the scenarios the vehicle faces.

Three concrete payoffs justify the complexity:

• Prediction quality in interaction. In multi-agent scenes, the marginal distribution of others' motion depends on the self-agent's plan. A model that conditions others' behavior on the self-agent's intended actions makes fundamentally different (and better) predictions than one that does not.
• Communication and cooperation. Handing someone a cup requires modeling what they know about the handover and when they will close their hand. That is a belief-tracking problem before it is a motor problem.
• Safety. Many failures of deployed agents come from under-modeling people: treating a human as a static obstacle produces plans that are technically feasible and socially disastrous. Over-modeling has a mirror-image failure, which we return to in Limitations.

The first payoff deserves emphasis. When another agent materially responds to the focal agent, the environment's transition distribution depends on the focal agent's behavior. Omitting that dependence then structurally misspecifies the interaction model rather than merely adding observation noise. Weakly coupled scenes and short horizons may still admit a useful simpler approximation, so the modeling depth should follow the sensitivity of the decision to the other agent's response.

There is also an introspective payoff. A self-model that includes "what others can observe about me" supports epistemic behaviors like signaling, feinting, and privacy, and it is the seed of the social self. Notice that this closes a loop with the beginning of the chapter: the agent-environment boundary, introduced as a private modeling choice, has a public counterpart, since other agents maintain their own models of what lies inside ours. An agent that represents the fact that it is in someone else's environment model has crossed from modeling the world to modeling itself as part of the world. We will revisit agent models at foundation scale in Part IX: Foundation and World-Action Models. Some foundation models display theory-of-mind-like behavior on particular benchmarks, but performance is uneven, and neither the underlying mechanism nor the construct validity of those benchmarks is settled.

To make everything concrete, we will build a miniature of the full chapter in code. The body schema is a neural network fit to self-generated calibration data. The learned model itself consumes only sampled joint configurations and measured end-effector poses. Analytic kinematics, Jacobians, and workspace circles are retained separately as evaluator-only ground truth, which a real experiment would replace with calibration fixtures or held-out measurements. The Jacobian comparison tests one local derivative, the reachability raster estimates positional capability, and the corridor observer performs level-1 Bayesian goal inference on a behavioral stream.

The agent is a two-link planar arm with link lengths $L_1 = 1.0$ and $L_2 = 0.8$, shoulder at the origin, joint angles $q = (q_1, q_2)$, and end-effector position

where:

• $x, y$: the Cartesian coordinates of the end-effector position in the plane
• $L_1, L_2$: the lengths of the first and second links
• $q_1$: the shoulder angle, measured from the horizontal axis of the world frame
• $q_2$: the elbow angle, measured relative to the first link

Each term projects one link onto the axis: the first link at angle $q_1$, the second at the accumulated angle $q_1 + q_2$, because $q_2$ is measured relative to the first link rather than to the world.

The agent does not get this formula. It receives self-generated calibration pairs: joint configurations from proprioception and measured Cartesian end-effector poses from an external tracker, calibrated camera, or other pose sensor. Real robots do not generally obtain Cartesian end-effector position from proprioception alone. From these pairs we will:

• Fit a neural forward-kinematics model: the learned body schema.
• Extract a Jacobian from the learned model by finite differences and compare it with the analytic one at one posture, testing local derivative quality without claiming closed-loop control performance.
• Compute a reachability map by sampling joint configurations and rasterizing end-effector positions: the positional affordance map, which should reveal the annulus with its inner hole.

Then, for theory of mind, we simulate a second agent in a one-dimensional corridor who walks toward one of two hidden goals, and build an observer that performs level-1 goal inference with Bayes' rule. The posterior should rise toward the selected goal; observing the agent stop at that goal makes the posterior exactly one because the alternative hypothesis assigns zero probability to that event. In this seeded run the posterior is already close to one before the stop, so the final update is mathematically decisive but visually small. The example is small enough to reason about exactly: its likelihood ratios, geometric bounds, and posterior values can be checked by hand.

The first block defines the true (hidden from the agent) kinematics and the self-generated calibration-data loop. Everything here is small enough to run on a laptop CPU in seconds. The separation between the definition block and the agent's data matters conceptually: the function forward_kinematics plays the role of the unmodeled physical body, and every subsequent step treats it as a black box that maps joint configurations to measured end-effector positions.

Read the two behaviors together. Before either goal is reached, a step toward position 9 multiplies the posterior odds for goal 9 by four, while a step away divides them by four. The net evidence in this seeded run favors goal 9. When the agent stops at 9, the goal-1 hypothesis assigns zero likelihood to that event and the posterior becomes exactly one. It moves only from about $0.999$ to $1.0$, so "exact" matters more than "dramatic." This is the core level-1 inference loop, but its conclusion is only as sound as the behavioral model. An obstacle, hesitation, or feint can make observed motion depart from the assumed policy and drive the same Bayesian update toward the wrong goal.

The key parameters for the self-model implementation are:

• hidden_layer_sizes: Widths of the multilayer perceptron's hidden layers. Two layers of 64 units give enough capacity for the smooth two-dimensional kinematic map without overfitting the 2,000 babbling samples. The sizing logic is worth internalizing: capacity should scale with the complexity of the map being learned and the amount of excitation data available, and both are modest here, so a modest network is correct.
• max_iter: Maximum optimization iterations for the schema fit. A large budget lets the optimizer converge on this small dataset. On larger problems, more optimization time may improve the schema but delays its use, so the budget should follow the deployment constraint.
• random_state: Seed for the network's weight initialization, which keeps the learned schema reproducible across runs. Reproducibility matters pedagogically here because we quote specific error figures; in a deployed system the equivalent concern is variance across training runs, since a self-model that varies run to run is a self-model the agent cannot fully trust.
• n_train: Number of random motor-babbling configurations used to fit the schema. Coverage of the joint space, not the count alone, determines where the schema can be trusted. A larger sample drawn only from one region of joint space would yield a locally excellent, globally unreliable schema, which is the excitation lesson from System Identification restated in self-model terms.
• eps: Step size for the central finite differences that extract the Jacobian from the learned model. Larger steps increase truncation error, while very small steps amplify floating-point and evaluation noise. The useful range depends on local smoothness, arithmetic precision, and any stochasticity in model evaluation. In this example, model approximation error dominates across a broad epsilon range, so the chosen value should be read as an empirical error-budget choice rather than a universal optimum.
• grid_res: Workspace cell size for the reachability raster. Finer grids resolve the annulus boundary more sharply but require denser sampling for coverage. This parameter couples directly to the affordance map's reliability: at fixed sample count, refining the grid converts boundary cells from "probably reachable" to "unresolved," which is honest but less useful, and the choice is ultimately a decision-theoretic one about how much boundary detail the planner needs.

Self-models are especially amenable to validation because the agent can choose some calibration actions and compare predictions with dedicated sensors. That advantage is bounded by safety, coverage, and sensor quality. Each layer still has characteristic failure modes.

The boundary choice is invisible when it is right and expensive when it is wrong. Place too little inside the self, for example by excluding a held tool, and the environment model must repeatedly infer action-coupled contact behavior. Place too much inside, such as the entire scene relative to the gripper, and the "self-model" stops transferring across tasks. In learned systems the boundary is implicit in which latents receive direct action inputs. A model that leaks policy information into environment latents may simulate its training distribution well and fail under interventions. Testing on shifted or off-policy action sequences is therefore a useful stress test for boundary choices. The symptom is not specific to boundary misallocation, however: poor coverage, optimization error, and other misspecification can cause the same failure, so follow-up interventions are needed to identify the cause.

Body schemas learned from data inherit the coverage of the agent's self-exploration. A schema trained near a nominal posture can be confidently wrong at extreme configurations. Calibration drift is the chronic version: gears wear, payloads change, temperatures shift, and a schema that was accurate at deployment can silently stop being so. The resilient-robotics result works because disagreement between prediction and sensing triggers a new identification cycle; long-lived embodied agents need an equivalent monitoring policy, and knowing when to distrust the self-model is as important as the model itself. Finite-sample affordance maps have a different evidential asymmetry. An observed success is a positive witness for that state-skill pair, whereas an unobserved success is not proof of impossibility. Negative conclusions require assumptions about exploration coverage and policy competence; dense sampling helps but does not supply those guarantees by itself. A planner that treats every empty cell as unreachable will forgo options, while one that ignores certified constraints will waste effort on hopeless actions. The corridor experiment gives a toy version of the same caution: its posterior reached certainty because the behavioral model was exactly right, while the same update under a wrong model could produce confident error.

Other-agent models carry the deepest limitations. Level-1 goal inference, like our corridor example, assumes the behavioral model is right; a real person sometimes moves away from their goal to avoid something, and the Bayesian update then confidently attributes the wrong goal. Nested reasoning compounds model error: to run level-2 inference, the agent must have a model of the other agent's model of itself, and errors multiply through the recursion. There are symmetric social failure modes: under-modeling people as physics produces safe-looking plans that are unusable around humans, while over-attributing intention (a learned model that sees goals in sliding doors and wind-blown bags) wastes computation and can produce bizarre avoidant behavior. Finally, theory-of-mind competence is hard to evaluate, because both the latent mental states and the ground truth for "understanding" are constructs; we will return to this in Part XI: Evaluation and Understanding. The evaluation problem is worth a closing note here, because it interacts with the four-way quality split from the opening: a model can pass behavioral theory-of-mind benchmarks, predictive fidelity on human trajectories, while its internal latent states bear no interpretable relationship to goals or beliefs, raising representation-quality questions the benchmarks never touch. Deciding how much that matters depends, once again, on whether the downstream use is prediction or explanation, and the field has not settled the answer.

The practical payoffs of these ideas are concrete. Model-based robot controllers rely on kinematic or dynamic models of the controlled system; these overlap with this chapter's broad body-schema concept, whether they are analytic, learned, or distributed across components. Learned and monitored schemas can turn some forms of damage into a re-estimation problem. Affordance models can make manipulation more tractable by pruning implausible actions and proposing useful contacts or skills, alongside geometric planning, optimization, feedback control, and learned policies. Models of other agents' trajectories, goals, or beliefs can improve crowd navigation and cooperation when interaction makes simpler reactive methods inadequate.

This chapter developed four layers of agent-relative representation, each building on the last.

• The agent-environment boundary is a modeling decision, guided by direct controllability, direct observability, and transfer across tasks. In latent world models, action coupling is useful evidence about which variables belong to the body, but it does not determine the boundary by itself. Good training-policy performance followed by failure under shifted actions can be one symptom of a poor split; off-policy testing is a useful but nonspecific stress test that requires follow-up diagnosis.
• Body schemas are action-oriented models of the agent's embodiment. They can be learned from self-generated data, checked against proprioception, and extended to tools through estimated tool parameters. Point predictions alone are insufficient when a controller consumes derivatives, and a pointwise Jacobian check is not a substitute for closed-loop validation.
• Capabilities describe feasible outcomes; in this chapter, affordances are agent-, skill-, outcome-, and horizon-relative success predictions. They trade generality for cheaper evaluation and can prune actions when their calibration and negative-evidence assumptions are explicit.
• Responsive agents can motivate latent goal or belief variables in the environment dynamics. Theory-of-mind-style inference updates those hypotheses from behavior, but its conclusion depends on the behavioral likelihood and the chosen recursion depth.
• Self-models can be checked directly through action-sensor residuals, but their validity still depends on exploration coverage and can drift with wear; recursive other-agent models add their own compounding errors.

The body schema tells the agent where it is; the next question is where everything else is. The following chapter, Geometry, Mapping, and 3D/4D State, extends the self-centered reachability map into full spatial representations of the environment: geometric state, maps built and maintained over time, and the 3D and 4D (space-time) representations that modern world models increasingly adopt.

Ready to test your understanding? Take this quick quiz to reinforce what you've learned about self-models, body schemas, affordances, and theory of mind.