Self-Models, Body Schemas, and Affordances › Summary

This chapter developed four layers of agent-relative representation, each building on the last.

• The agent-environment boundary is a modeling decision, guided by direct controllability, direct observability, and transfer across tasks. In latent world models, action coupling is useful evidence about which variables belong to the body, but it does not determine the boundary by itself. Good training-policy performance followed by failure under shifted actions can be one symptom of a poor split; off-policy testing is a useful but nonspecific stress test that requires follow-up diagnosis.
• Body schemas are action-oriented models of the agent's embodiment. They can be learned from self-generated data, checked against proprioception, and extended to tools through estimated tool parameters. Point predictions alone are insufficient when a controller consumes derivatives, and a pointwise Jacobian check is not a substitute for closed-loop validation.
• Capabilities describe feasible outcomes; in this chapter, affordances are agent-, skill-, outcome-, and horizon-relative success predictions. They trade generality for cheaper evaluation and can prune actions when their calibration and negative-evidence assumptions are explicit.
• Responsive agents can motivate latent goal or belief variables in the environment dynamics. Theory-of-mind-style inference updates those hypotheses from behavior, but its conclusion depends on the behavioral likelihood and the chosen recursion depth.
• Self-models can be checked directly through action-sensor residuals, but their validity still depends on exploration coverage and can drift with wear; recursive other-agent models add their own compounding errors.

The body schema tells the agent where it is; the next question is where everything else is. The following chapter, Geometry, Mapping, and 3D/4D State, extends the self-centered reachability map into full spatial representations of the environment: geometric state, maps built and maintained over time, and the 3D and 4D (space-time) representations that modern world models increasingly adopt.