Locomotion, Humanoids, and Navigation

Imagine three tasks: a quadruped meeting an unfamiliar slope, a humanoid balancing while reaching, and a wheeled robot navigating with a map built earlier. These are illustrative scenarios, not reported trials. They differ in embodiment and horizon, but each requires predicting how actions change the robot's state and surroundings before choosing what to do.

Earlier chapters introduced state, action, partial observability, system identification, and planning. Part V: World-Model Architectures developed learned predictive representations; Part VII: Planning and Agency used them for action selection. Here we ask what those ideas require when a robot makes and breaks contact with terrain while moving toward a goal.

Locomotion and navigation put different pressures on a model. A legged controller must account for changing contact modes and uncertain surface properties; a navigator also needs spatial memory and a way to evaluate routes. Errors can compound over repeated predictions, especially when a planned footfall or route crosses a poorly modeled region. The fixed-base manipulation examples in the previous chapter have different actuation and contact assumptions, but neither domain is categorically easier.

This chapter follows body dynamics and terrain interaction, whole-body control, then spatial memory and goal navigation. We finish with cross-embodiment and sim-to-real transfer. The worked models are deliberately small: they show how contact modes, terrain uncertainty, and map uncertainty enter prediction and planning without claiming to reproduce a full robot stack.

We begin with the mechanics of a floating base so the later control equations have concrete state and force variables. Planning then connects those predictions to actions. Navigation adds a map and longer-horizon decisions; it does not replace the shorter control loop beneath it.

Terrain uncertainty appears at several levels. At the physics level, unknown height or friction changes predicted transitions. At the control level, it can change which candidate motions satisfy contact limits. At the navigation level, it can change the cost or feasibility of a route. A single aggregate prediction score can miss these differences, so we evaluate the quantities each decision uses.

A physical transition model is one component of that planning system:

where:

• $s_t$: your robot's physical state at time $t$ (base pose, joint angles, and velocities).
• $a_t$: the control action at time $t$ (joint torques or desired joint positions).
• $f$: the physical transition model mapping current state and action to next state under an assumed body, terrain, and contact mode. Goals, route costs, and spatial memory belong to the task or map model used by the planner; they need not be components of $f$.

A too-smooth terrain estimate and an overestimated friction coefficient might fit familiar trajectories yet fail on a different surface. Aggregate one-step error alone may not identify which component is responsible. Contact-specific tests, terrain-parameter checks, and closed-loop outcomes provide different evidence about the model.

A typical legged robot has a floating base with six local configuration coordinates: three for position and three for orientation. Joint torques do not directly actuate those coordinates; gravity and external contact forces affect base motion. This differs from the fixed-base arm examples in the previous chapter, though mobile manipulators can also have moving bases. Inverse kinematics can propose a pose, but dynamic feasibility still depends on contact, torque, and friction limits.

The equations of motion for a floating-base robot with $n_j$ joints describe the balance of inertial forces against actuation and contact forces. The floating-base rigid-body dynamics in standard form are:

where:

• $q \in \mathbb{R}^{6 + n_j}$ is a local coordinate representation of base pose plus joint angles; three orientation coordinates require a local chart.
• $M(q)$ is the mass matrix, coupling joint and base accelerations.
• $C(q, \dot{q}) \dot{q}$ is the Coriolis and centrifugal force vector.
• $G(q)$ is the gravity force vector.
• $S = [0_{n_j \times 6} \mid I_{n_j}]$ selects the actuated joints, so $S^\top \tau$ applies joint torques and nothing to the six unactuated base degrees of freedom.
• $\mathcal{C}$ is the set of currently active contacts, $J_i$ is the contact Jacobian, and $f_i$ is the contact force at contact $i$.
• $J_i^\top(q) f_i$ is the generalized force induced by contact $i$, including a wrench on the floating base.

The left-hand side of the equation collects inertial, velocity-dependent, and gravitational terms. On the right, $S^\top\tau$ applies commanded joint torques, while $J_i^\top f_i$ represents generalized contact forces, including forces on the floating base. Contact forces are not independent commands in this formulation: they must be feasible consequences of motion, geometry, and contact constraints.

The important observation is that the base has no actuation term in the right-hand side: joint torques act on it only through the limbs and their contacts. Gravity still accelerates the center of mass in flight, and active contact forces can accelerate and rotate the base. Locomotion uses joints to push the ground so that the ground pushes back in a direction that moves the robot toward its goal. You must also respect actuator torque limits, joint velocity limits, and the available friction at each contact, or the base motion you plan will not be achievable. These limits separate a mathematically valid trajectory from one the hardware can execute, and they reappear throughout the whole-body control section below.

Two things follow. First, the base motion is not directly commanded; it emerges from gravity, internal body motion, and contact. A world model of locomotion must account for contact forces as well as joint angles. Second, if no contact is active, the robot is in free flight and $M(q) \ddot{q} + C(q,\dot q)\dot{q} + G(q) = S^\top \tau$. Joint motion can reorient the torso relative to the limbs, as in the "falling cat" effect. But internal torques cannot accelerate the whole-body center of mass. With gravity as the only external force, that center of mass follows a ballistic trajectory; its linear momentum changes under gravity, not under internal actuation.

The falling-cat intuition is worth dwelling on because it recurs across the whole chapter. A gymnast who has left the ground cannot accelerate her center of mass upward using only internal motion, but she can twist her limbs to reorient her torso. In the same way, a legged robot in the flight phase of a hop can tuck or extend its legs to change its orientation for landing, but it cannot steer its center-of-mass trajectory without an external force. For the vertical hopper below, flight is simple ballistic motion; predicting a full robot's free-flight orientation and landing configuration can still be difficult. Stance adds large, state-dependent contact forces, so splitting predictions by contact mode is useful.

Another structural feature is that legged locomotion often has distinct contact modes. Touchdown, liftoff, and slip can change the applicable force law; a rigid impact may reset velocity, whereas a compliant-contact model can change forces continuously. A predictor may need both within-mode dynamics and the timing of transitions. The consequences of mistiming depend on the model and task: a wrong contact mode can produce a large rollout error, but neither a discontinuous state jump nor error growth is inevitable.

Consider a one-dimensional vertical hopper, the simplest instructive example. Let $z$ be the height of the body, $v$ its vertical velocity, and $h(x)$ the terrain height at horizontal position $x$. The dynamics switch between two modes: ballistic flight under gravity, and stance with a compliant ground reaction force. Written as a piecewise law:

where:

• $z$: the height of the body, with $\ddot{z}$ its vertical acceleration.

• $g$: the gravitational acceleration.

• $m$: the body mass.

• $F(z, \dot{z})$: the toy spring-damper force in stance, given by $F(z, \dot{z}) = -k\,(z - h(x)) - b\,\dot{z}$.

• $h(x)$: the terrain height at horizontal position $x$.

• $k$: the ground stiffness, and $b$: the ground damping. The transition rules are equally important:

• Flight $\to$ stance when $z \leq h(x)$ and $\dot{z} \leq 0$.

• Stance $\to$ flight when $z > h(x) + \epsilon$ in this toy's hysteretic mode rule.

This simple system illustrates gravity, a terrain-dependent force, event-triggered mode switches, and a trajectory shaped by switch timing. It is deliberately not a physically exact unilateral contact model: the unclamped spring-damper can briefly pull downward while the mode remains stance, whereas a real ground contact cannot pull the foot toward the surface. Extending to a biped also requires multiple contacts, surface normals, friction, impact handling, and active joint torques. The hopper isolates the hybrid timing problem without claiming to capture every difficulty of locomotion.

To build intuition for the one-dimensional case, note that the flight mode is a simple kinematic equation $\ddot{z} = -g$ whose solution is a parabola, and the stance mode is a mass-spring-damper system:

where:

• $m\ddot{z}$: the inertial force on the body.
• $-mg$: the gravitational force.
• $-k(z - h(x))$: the elastic restoring force from the compliant ground, proportional to how far the body has compressed below the terrain height $h(x)$.
• $-b\dot{z}$: the damping force opposing vertical velocity.

In this toy, a state-dependent guard switches between flight and stance, so the trajectory is piecewise smooth and guard timing matters. Near a transition, a small height or velocity error can select the wrong mode. The resulting rollout then uses the wrong force law until its mode estimate is corrected; its error may be substantial, but its magnitude and direction depend on the trajectory. Guard accuracy and within-mode accuracy both matter, and neither is universally the dominant engineering problem.

Here is where locomotion world models differ most sharply from generic sequence models. The transition function depends on variables such as terrain height and friction that are not part of the robot's proprioceptive state. The robot senses terrain through a camera, LiDAR, a depth sensor, or proprioceptive feedback when its feet make contact. Predictions over terrain it has not yet stepped on are inherently uncertain. Put differently, the hopper's rollouts above assumed the terrain function $h(x)$ was known; in reality $h$ is one of the quantities the robot must estimate, and it changes from one part of the world to the next.

We can make this formal by writing the environment as a partially observable process. Let $\theta$ denote the terrain parameters (per cell: height, slope, friction, compliance), $s_t$ the robot's physical state, and $o_t$ its observations. A latent-parameter model has a transition law conditioned on terrain and an observation law conditioned on both state and terrain:

where:

• $\theta$: terrain parameters (per-cell height, slope, friction, compliance) that are fixed but unknown
• $s_t$: robot's physical state (base pose, joint angles, velocities)
• $a_t$: control action (joint torques or desired joint positions)
• $o_t$: observation (proprioceptive sensors and exteroceptive sensors such as camera or LiDAR)

The controller maintains a belief $b_t(\theta)$ over terrain, in the sense of the posterior introduced in the filtering sections. A landing's impact and resulting body motion can be informative; between contacts, exteroceptive sensing can also update the belief. This is the belief-state framework from Part II, Ch 2: Bayesian Filtering and Belief States, applied to terrain alongside robot-state estimation. Here $\theta$ is fixed but unknown over the modeled region; $s_t$ is robot pose, joints, and velocities; $a_t$ is a commanded torque or joint target. The conditional transition may itself be deterministic or stochastic. Even if it is deterministic for a fixed $\theta$, marginalizing over $b_t(\theta)$ makes the controller's prediction uncertain. A controller can plan against a nominal terrain estimate or average over belief samples; the nominal MPC formulation below illustrates the former. In practice terrain and body state are often estimated jointly: a footstep on an unexpectedly soft patch informs both the surface model and the body's velocity estimate.

The terrain remains uncertain even when the robot's pose is well estimated. If terrain were fully observed, one could append it to the robot state and recover a standard MDP. The latent-parameter framing clarifies what each observation contributes: contact measurements are local and direct but can be distorted by slip or compliance; exteroception covers terrain ahead but depends on visibility and geometry. An estimator can combine both, updating its local surface estimate after contact while using cameras or depth sensors to anticipate terrain ahead.

We can see a toy belief over terrain narrowing as observations arrive. In the following synthetic measurement model, each contact supplies an independent, unbiased Gaussian observation of one fixed local height. With the stipulated flat prior, posterior standard deviation is $0.12/\sqrt{n}$ after $n$ observations. Real contacts can be biased or correlated, so they need not follow this curve.

The practical lesson is conditional. Ignoring uncertainty in a terrain estimate can make a predicted step overconfident; omitting task-relevant terrain information can increase transition error or hide useful foothold choices. Neither outcome follows for every controller or gait. Perceptive locomotion systems can combine proprioceptive feedback with exteroceptive information about terrain ahead, as in Miki et al.'s quadruped work below. Their sensing and control update rates depend on the implementation; both information streams can contribute to feedback and prediction.

There is a middle ground between a full physics-based simulator and an end-to-end neural network. Some systems learn a terrain-related latent from recent proprioceptive history and condition a policy on it. Such an encoder approximates information in a terrain belief $b_t(\theta) = p(\theta \mid o_{1:t}, a_{1:t})$, although its latent need not be a calibrated posterior. Lee et al. (2020), "Learning Quadrupedal Locomotion over Challenging Terrain" trained a proprioceptive student policy with a temporal convolutional network over observation history, supervised by a privileged teacher. Miki et al., 2022, "Learning robust perceptive locomotion for quadrupedal robots in the wild" combined proprioception with exteroceptive height samples from an elevation map constructed from onboard sensor data; an attention-based recurrent belief encoder learned how to use imperfect perception. Miki et al. did not learn the elevation map itself in the policy. These examples show different ways to condition locomotion on inferred terrain information, not a single explicit probabilistic world-model architecture.

Such architectures can separate an estimator from a policy, making the interface inspectable when it is explicitly defined. A history encoder may infer terrain-related properties from joint and body measurements, commands, and contact responses; which signals are useful depends on the sensors and actions. Similar histories can arise on different surfaces, and terrain can change within an episode, so an inferred latent is not automatically an identifiable or calibrated terrain parameter.

When terrain is not directly observable but affects a trajectory, explicit parameter estimation and recurrent history encoding are two options; exteroceptive sensing and hybrid estimators add others. An explicit estimate can be easier to inspect and calibrate, while a recurrent latent can retain useful history without a hand-specified terrain variable. Neither has a general advantage in data efficiency or failure rate. Those properties need comparison on the intended robot, sensor suite, and terrain.

The tradeoff can be read as a choice of where uncertainty lives. Explicit estimation keeps the terrain posterior separate and lets the planner propagate it forward, whereas recurrent absorption pushes the uncertainty into the hidden state, which is harder to inspect but avoids hand-designing a terrain model. Which choice is better depends on what you intend to do with the uncertainty. If you plan to plan conservatively, an explicit posterior that can be queried for a confidence bound is invaluable. If you only need a policy that acts well, an absorbed hidden state may be all you need, and the representational freedom it allows can be an advantage rather than a liability.

Let us make the hybrid, terrain-dependent dynamics concrete by simulating the simple 1D hopper. The simulator below uses a fixed two-millisecond explicit step and a spring-damper guard rule. We do not establish numerical convergence or model a rigid impact; the goal is to expose mode structure, not to match a production simulator.

Notice how much the terrain function enters even this one-dimensional model. Its stance force depends on displacement relative to terrain height at the contact, so height error changes the predicted force and acceleration. A full legged robot can have similar coupling at its active contacts; joint terrain and body-state estimation is one possible response, not a requirement for every controller.

We have identified quantities a locomotion model may need to predict. The next question is how a controller uses them to choose actions. Whole-body control is one structured answer; direct learned policies and other feedback designs are also possible.

"Whole body" refers to coordinated motion across the torso, relevant joints, and active contacts. A controller may jointly choose foot placement, feasible contact forces, torso orientation, and limb posture; the exact decision variables depend on the robot and task. The coupling matters: changing foot placement affects contact forces and torso acceleration, which in turn affect posture and where the feet can reach next. Optimizing those pieces independently can miss a feasible or stable coordinated motion.

One formulation is nominal model predictive control (MPC) over the hybrid dynamics. Recall from Part VII, Ch 1: Sampling-Based Planning and Model Predictive Control that MPC at each control step solves a finite-horizon optimization problem, applies the first action, then re-solves. Here the controller plans with one terrain estimate $\hat\theta_t$ and minimizes stage costs subject to dynamics, actuator limits, and contact/friction constraints. This formulation does not propagate the full terrain belief; scenario or belief-space MPC would account for that uncertainty explicitly:

where:

• $H$: planning horizon (number of steps)
• $a_t$: control input at time step $t$ (e.g., joint torques). A planner may also optimize feasible desired contact forces, but a lower-level controller must realize them through the robot's actuators and contacts.
• $s_t$: full robot state (generalized coordinates and velocities)
• $f(s_t, a_t, \hat\theta_t)$: hybrid dynamics evaluated at the current terrain estimate $\hat\theta_t$
• $c(s_t, a_t)$: stage cost at time $t$ (e.g., tracking error, control effort)
• $c_H(s_H)$: terminal cost at the horizon
• $\hat{s}_0$: estimated initial state (from state estimator)
• $\mathcal{A}(s_t)$: set of admissible actions at state $s_t$ (respecting actuator limits)
• Contact constraints: constraints ensuring feet are on the ground and forces lie in friction cones

The objective sums costs over a horizon. A major difficulty lies in the constraints: the nominal dynamics $s_{t+1} = f(s_t, a_t, \hat\theta_t)$ are hybrid, so contact-mode choices create different feasible regions. The admissible-action constraint encodes hardware limits, while contact and friction constraints restrict forces and motion. A low-cost trajectory that violates one of these constraints is not executable as planned. A trajectory feasible for $\hat\theta_t$ can still fail on uncertain terrain, which is why robust or belief-aware variants matter.

Three choices make this formulation useful for the legged example, though related choices also appear in manipulation:

• $f$ is the hybrid rigid-body dynamics above. It contains contact events, and small differences in their timing can produce large differences in future state on trajectories sensitive to those events.
• $c$ can penalize center-of-mass (CoM) tracking error, footstep error, body orientation, control effort, or torque rates; its form depends on the task.
• At a sticking contact, the force $f_i$ must satisfy a unilateral normal-force condition and a friction cone $\{f : \|f_{\text{tangential}}\| \leq \mu f_{\text{normal}}, f_{\text{normal}} \geq 0\}$. A force outside that cone cannot be sustained as a sticking contact. Force feasibility also depends on contact location, joint kinematics, and actuator torque limits; being inside the cone alone does not make a whole-body plan executable.

The cone is easiest to read directly in force coordinates, where it is the wedge bounded by the two lines $f_{\text{tangential}} = \pm \mu f_{\text{normal}}$. Admissible contact forces sit inside the wedge, and slipping forces sit outside it.

Solving the full hybrid optimization at a high control rate can be expensive; feasibility depends on the model, horizon, solver, and hardware. Three common strategies simplify different parts of the problem.

The first is contact-implicit trajectory optimization, exemplified by Posa, Cantu, and Tedrake (2014), "A Direct Method for Trajectory Optimization of Rigid Bodies Through Contact". Their formulation optimizes motion and contact forces together with complementarity constraints rather than fixing a contact schedule in advance. This mathematical program with complementarity constraints can be numerically difficult. Related contact-implicit formulations may be used in receding-horizon control, but the cited work is trajectory optimization, not a claim that a particular smoothing parameter is always used in MPC.

The second is contact-scheduled optimization, used in several legged-control systems, including MIT Cheetah-style controllers; Crocoddyl is a library that can formulate such problems. A gait pattern or higher-level planner supplies a candidate contact schedule, and continuous optimization then solves for motion and forces under it. This avoids searching over every schedule in each solve, but a poor schedule can rule out an otherwise feasible motion. Feedback can revise timing or the schedule when conditions change. These examples do not establish how prevalent the approach is across deployed quadrupeds.

The third category needs a distinction. A learned dynamics model can be used inside a model-predictive planner; TD-MPC, from Part VIII, Ch 5: TD-MPC and Control-Centric Representations, is an example that uses learned latent dynamics, rewards, and value estimates for short-horizon planning. A direct policy trained by imitation or reinforcement learning may instead map observations to actions without planning through an explicit learned model. Analytic and learned models offer different inductive biases and failure modes; neither has a general guarantee of better behavior on hardware.

The two optimization strategies and learned-model MPC share a predict–optimize–execute–replan loop, but direct learned policies need not. The relevant comparison is therefore between how a controller represents dynamics and contacts, how it selects actions, and how it uses feedback, not an assertion that all reinforcement-learning controllers run MPC.

One reason whole-body control can be tractable is that the net motion of the center of mass (CoM) and its angular momentum can be expressed in a smaller space than the full $6+n_j$ robot coordinates. Internal forces cancel from the whole-body balance equations. In differential form:

Here $\mathbf g$ is the gravity-acceleration vector, $r_i$ the contact point, and $\tau_i^{\text{contact}}$ any direct contact moment, which is zero in a point-contact model. Internal forces do not appear explicitly in these net equations. That does not make joint motion irrelevant: kinematics, torque limits, and available contacts determine which centroidal trajectories the robot can realize. A low-dimensional planner can propose CoM and momentum behavior, while a whole-body controller checks and realizes it through the joints.

The centroidal model is a useful low-dimensional planning surrogate. Its CoM dynamics depend on gravity and contact forces, while the contact sequence remains hybrid. A further simplification is the linear inverted pendulum model (LIPM): with constant CoM height, a point support, and negligible rate of centroidal angular momentum about the CoM, the horizontal equation becomes linear and foot placement becomes a planning input. These assumptions make the LIPM useful for illustration, not a full robot simulator.

The first equation is Newton's second law for the whole-body CoM: gravity and the vector sum of contact forces determine its acceleration. The second is the angular-momentum balance about the CoM: contact forces acting at moment arms $r_i-p_{\text{CoM}}$, plus any direct contact moments, change $L$. In the LIPM special case, the CoM height is fixed at $z_0$, support is a point at $p_{\text{foot}}$, and the centroidal angular-momentum rate is neglected. The horizontal dynamics then become $\ddot{p}_x = (g/z_0)\,(p_x-p_{\text{foot}})$, where $g$ is the positive magnitude of gravitational acceleration. Changing $p_{\text{foot}}$ between steps supplies the control input in the example below.

The instability and the role of foot placement are visible in a short numerical integration of that ODE. With a fixed foot, the CoM accelerates away from the contact point. In the plotted feedback run, re-placing the foot at the capture-point estimate $p + \dot{p}/\omega$ every step interval brings the CoM toward a bounded position.

For a fixed foot placement, the linear LIPM has a closed-form CoM solution; choosing the next $p_{\text{foot}}$ changes that solution. A controller can use a centroidal planner for a candidate CoM and momentum trajectory, then a whole-body inverse-dynamics layer to seek joint torques that realize it. The layers must be checked together: joint reach, torque limits, and contact availability can invalidate the candidate. Their update rates depend on the robot and implementation, not on a universal planner-versus-controller frequency split.

The LIPM gives a concrete interpretation of foot placement. With a fixed point support, its horizontal CoM dynamics are unstable: displacement from the support tends to grow. Repositioning the support can change that trajectory. The capture-point-style rule in the code illustrates one feedback choice, not a general solution for every gait, contact geometry, or robot.

Analytic whole-body controllers use approximate body dynamics, parameter estimates, and contact assumptions. Feedback can tolerate some mismatch, but unmodeled compliance, soft terrain, or damage may degrade performance. Learned corrections and adaptation are possible responses; their value must be tested against a well-tuned analytic baseline.

One pattern is residual learning: fit a model of the mismatch between analytic and observed dynamics and use it alongside the analytic model. This resembles system identification with a flexible model class (Part II, Ch 3: System Identification). Training data can come from simulation, hardware, or both, and the residual must be evaluated on the target states and contacts. An unconstrained residual does not automatically fall back to physics outside its training data; that behavior requires an explicit gate, bound, or uncertainty-aware design.

Another pattern learns dynamics or a policy from data. The Dreamer family from Part VIII, Ch 3: The Dreamer Family learns a recurrent latent model from experience and trains an actor and critic using imagined trajectories. This is distinct from privileged teacher–student locomotion training: some methods train with simulator-only terrain information and then distill or adapt a controller that uses onboard sensing. Neither approach implies that every humanoid or quadruped learns a single latent world model of body and terrain, and a privileged teacher does not guarantee successful extraction from noisy sensors.

The distinction is about inductive bias, not a strict body-versus-terrain split. An analytic backbone can preserve useful structure, but a learned residual may still produce physically implausible outputs unless constrained. An end-to-end model can learn unmodeled effects and can also incorporate physical structure. Which combination works best depends on the robot, data, and validation regime.

Whole-body control can encounter several failure modes worth testing explicitly; their frequency and severity depend on the system.

• Contact mistiming. The controller expects a foot to be in contact at time $t$ and it is not, or vice versa. This happens when terrain height is misestimated, when the compliance of the surface is not modeled, or when the actuator bandwidth limits how fast the leg can extend.
• Friction cone violation. A requested tangential force may exceed the available sticking-friction bound, causing slip; whether the robot recovers depends on its feedback and remaining contacts. A model trained on one surface may misestimate this bound on another.
• Model exploitation. The optimizer finds a plan that is optimal under an inaccurate model but impossible in reality: for example, a step that relies on force that the model thinks is available but the actuator cannot produce. This is the canonical "model-based RL gone wrong" story from Part XII, Ch 1: Failure Modes and Model Exploitation.
• Compounding rollout error. Repeated model errors can accumulate over an open-loop horizon, though they can also contract under stable dynamics. The relevant horizon depends on the model, controller, and task.

The four failure modes interact rather than dividing neatly into physical and computational categories. Contact mistiming or slip may follow a poor terrain estimate, while model exploitation can end in a physical loss of balance. Compounding prediction error can also misrank otherwise feasible plans without making any action physically impossible. Better contact and friction estimates help, as does measuring where the predictive model is unreliable; a controller needs both kinds of checks because an unexpected contact changes the state from which the next prediction begins.

No single mitigation covers all four. Re-observation and replanning can reduce exposure to open-loop error, subject to sensing and computation limits. Contact and friction failures also require feasible constraints, calibration, and monitoring. An ensemble may reveal model disagreement, but disagreement is not a calibrated detector of every out-of-distribution state or model-exploitation risk. Feedback update rate and rollout horizon should be chosen from measured closed-loop behavior, not a universal short-horizon rule.

We test whether a learned hopper model can sustain the multi-step predictions that MPC would use. The experiment collects simulator transitions and fits a linear model of vertical dynamics; it does not optimize actions or close a control loop.

The takeaway is that one-step height prediction is small but not exact, including in flight: although the true flight law is linear, this fitted map pools flight and stance transitions. Recursive prediction diverges as small biases accumulate and the model fails to represent contact-mode changes. Re-solving MPC from new observations limits the time spent following an open-loop prediction, but the appropriate update rate and horizon depend on sensing, computation, and the robot's dynamics. This toy experiment establishes open-loop prediction error, not a safe control frequency.

The plotted divergence can reflect fitted-model bias and missed contact structure, but this experiment does not isolate their contributions. Bias need not grow smoothly, and a mode-timing error need not enter a different basin. Better data, explicit modes, contact-aware losses, uncertainty estimates, and timely feedback are candidate remedies; each needs testing against the failure it is meant to address.

Locomotion determines how the robot moves its body. Navigation determines where it should go. The two are coupled: the choice of gait depends on the destination and obstacles, and the choice of route depends on what gaits the terrain permits. In an autonomous mobile robot, the coupling is managed through a world model that includes a map: an internal representation of the environment that supports spatial reasoning over long horizons.

The map is not a luxury. A robot's sensors have finite range; a decision about where to go next must often be made based on information that left the field of view minutes ago. The environment is partially observable, and the belief state includes the robot's estimate of layout, occupancy, and semantics of the space. This is the navigation-specific instance of the POMDP belief state from Part II, Ch 1: Markov and Partially Observable Decision Processes and Part II, Ch 2: Bayesian Filtering and Belief States. In the navigation case the latent state includes the robot pose $x_t$, the map $m$, and often the motions of other agents, while the observations $z_t$ are range or image measurements and the actions $u_t$ are velocity commands or waypoints.

One reason to build a map is to support choices beyond the current sensor view. A purely current-observation policy cannot distinguish two locally identical hallways with different known destinations; stored spatial information can. An explicit map is one such memory structure, while a recurrent policy or other memory can also retain useful history. A probabilistic map may be part of the robot's belief over persistent layout, but maps need not always be represented as posteriors.

Several map representations appear in navigation. A metric occupancy grid or volumetric map assigns occupancy estimates to spatial cells, as in OctoMap. It can support local planning, but dense high-resolution coverage of a large environment can be costly. Sparse, hierarchical, feature-based, and graph representations have different storage and query scaling; SLAM does not imply a dense grid.

The second layer is topological: a graph whose nodes are landmarks or rooms and whose edges represent traversability, as in the classical work on topological SLAM and in Chaplot et al. (2020), "Neural Topological SLAM for Visual Navigation". Topological maps are scale-efficient and support long-horizon reasoning, but they require a mechanism to decide which observations become nodes. That mechanism is the crux: a topological map is only as good as its node placement, and getting the granularity right, coarse enough to be compact and fine enough to be informative, is a real design problem.

The third layer is semantic: object or place categories and their relations. Such information can provide priors for where a goal object might be, including when geometry is incomplete. Vision-language models may supply candidate semantic priors, but they cost computation and can be wrong; a category association is not an observation of what lies behind a closed door.

Different systems combine these representations according to task and scale. A metric map can help resolve nearby obstacles; a topological graph can organize long routes; semantic labels can guide object search. None has a single exclusive function, and this chapter does not establish a universal most-robust combination. The useful mixture depends on localization quality, map updates, route length, and the cost of mistakes.

Simultaneous localization and mapping (SLAM) is a family of methods for jointly estimating a robot trajectory and a map from sensor data. One probabilistic formulation tracks a belief over current pose $x_t$ and map $m$:

where:

• $x_t$: robot pose at time $t$ (position and orientation)
• $m$: map of the environment (e.g., occupancy grid, landmarks)
• $z_{1:t}$: sequence of observations from time 1 to $t$
• $u_{1:t-1}$: controls applied between successive poses $x_1$ through $x_t$
• $b_t(x_t, m)$: belief over pose and map given all observations and controls

The observations through time $t$ and controls applied before time $t$ condition a joint belief over the robot's location and the surrounding world. This is a world model that is updated from data and queried by a downstream planner. The chicken-and-egg structure is why SLAM is hard: the robot needs its location to build a consistent map, and it needs a map to estimate its location. Joint inference addresses both unknowns together. Other indexing conventions are possible if the time interval attached to each $u_k$ is defined consistently.

SLAM systems make different structural choices. EKF-SLAM maintains a Gaussian approximation over pose and landmarks, with covariance updates that can be expensive as landmark count grows. Factor-graph and bundle-adjustment systems, including Cartographer and ORB-SLAM, optimize pose and measurement constraints. Their speed and loop-closure reliability depend on data association, graph size, solver, and implementation; graph structure alone does not guarantee either advantage.

Loop closures are one payoff of the graph formulation. Dead reckoning can let pose uncertainty accumulate along a trajectory; a valid, informative loop-closure constraint can reduce drift or uncertainty after optimization. A mistaken association may instead damage the estimate.

Neural implicit mapping can represent geometry with fields queried at spatial points; some systems combine such fields with explicit structures. Neither an explicit nor a learned map is automatically compact, cheap to query, complete, or calibrated in unobserved regions. Explicit maps can retain unknown cells or uncertainty, while learned maps may interpolate plausible geometry that needs validation. Their value for planning must be measured on the target scene and query workload.

A map supports goal-directed navigation, but localization, perception updates, and motion control remain necessary. Three illustrative planning strategies follow.

The first is geometric planning: run A*, D*, or a sampling-based planner over an occupancy grid or topological graph, then follow the resulting path. This classical approach remains useful where the map and travel constraints can be represented explicitly. The map is the spatial model, the planner computes a policy or route, and the objective may be path length or a risk-adjusted cost. Its explicit representation often makes a bad route easier to diagnose: one can inspect the map cells or graph edges that the planner used.

The second is learned navigation with a spatial map. Chaplot et al. (2020), "Learning to Explore using Active Neural SLAM," combine a learned map-and-pose module, hierarchical policies, and analytical path planning. They reported stronger exploration performance and sample efficiency than the particular end-to-end baselines they evaluated. That result supports explicit spatial structure as a useful inductive bias in their setting; it does not establish a universal advantage over recurrent or other map-free policies.

The third strategy uses learned spatial features. A network may write observation-derived features into cells or graph nodes and let a policy or planner query them. This extends temporal memory with spatial indexing (Part III, Ch 4: Temporal State and Memory). The cells may be deterministic latent features or an explicit uncertainty-aware belief; latent storage alone does not require Bayesian posterior planning or guarantee accurate geometry.

These strategies vary in how much geometry and uncertainty they expose. Explicit maps can make a route easier to inspect but can contain unnoticed estimation errors; learned features can help with ambiguous observations but may also hallucinate missing geometry. Storage, query cost, calibration, and transfer vary by implementation. A hybrid map is one option, not an established most-robust design for every navigation task.

Compared with a local gait controller, a navigation model often emphasizes two additional demands; a full locomotion system can also face them.

First, navigation can span far more time and distance than one gait cycle. Long-route planning therefore benefits from abstract state such as occupancy grids or topological maps. A fine-grained dynamics rollout over a long route would require many model steps and incur substantial computation and potential compounding error; a route planner can instead work at a coarser scale and leave local motion to the locomotion controller.

Second, some navigation actions gather information rather than shorten the immediate route. Turning a corner may reveal whether a hallway continues; that observation can improve a later route choice. Part VII, Ch 5: Active Perception, Dual Control, and Exploration develops this trade-off. Exploration is useful when the expected value of a better map outweighs its travel and sensing costs; it is not automatically the best choice on every task.

We now make the geometric planner concrete by building an occupancy grid, running value iteration on it, and extracting a path.

The path is not a straight line to the goal because the world model encodes the geometry of the environment. It follows open column 2 to row 22, then traverses row 22 to the goal, avoiding the interior walls and blocks. Value iteration is the classical answer, and it is instructive to note what is not happening: you are not learning a policy, you are computing a policy from a known map. The value iteration update is:

where:

• $V(s)$: the discounted return from $s$ under the best route in this deterministic grid, with reward $-1$ per step and zero terminal value.
• $\gamma$: the discount factor, with $0 < \gamma < 1$, weighting future costs relative to immediate ones.
• $s'$: a neighboring state reached by taking action $a$ from state $s$.
• $-1$: the per-step cost, a penalty charged for each step taken.

The max is over admissible neighboring states that are not occupied. The same value-iteration update can use occupancy estimates from a learned mapper if states and transitions are defined for them. Other learned-map systems use graph search, analytical planners, or policies instead; Chaplot et al.'s system is not this notebook's value-iteration loop. Here the map supplies spatial state and value iteration computes a route. A body MPC and a route planner can both be updated as observations change, but their models, actions, and replanning schedules differ.

Up to this point we have assumed a single robot and a single terrain. Real robots differ in mass, actuators, wear, and payload, and collecting failure-rich locomotion data on hardware can be costly or damaging. Simulation is therefore an important training and testing tool, raising the question of how a model transfers to physical hardware or another body.

Cross-embodiment is the study of world models and policies that transfer across morphologies: a quadruped that can be re-trained to walk on a different quadruped, a humanoid that shares a world model with a small biped, a manipulation policy that generalizes across gripper widths. Sim-to-real transfer moves a model or policy from simulation to physical hardware; it may keep the same embodiment or involve an embodiment change.

Cross-embodiment is plausible because robots share conservation laws and contact mechanics, even though their kinematics, actuators, observation spaces, and feasible contacts differ. A transferable representation can encode shared structure while retaining body-specific state and action interfaces. Whether joint training helps is an empirical question for the bodies and tasks involved.

Three gaps bite.

The dynamics gap is mismatch between simulated dynamics and the physical robot. Real joints may exhibit friction, backlash, stiction, saturation, thermal effects, and delay that a particular simulator approximates imperfectly. Mass and actuator parameters may also be uncertain. These errors can alter contact timing or forces, sometimes changing a stable predicted step into a stumble; the size of the effect depends on the robot, controller, and surface.

The perception gap is mismatch between simulated and real sensing. A simulator may omit depth holes, reflections, lighting changes, lens distortion, latency, or other noise; these effects can also be modeled or randomized. The importance of a particular gap depends on which sensors the controller uses and at what rates. Exteroceptive terrain estimates can be especially sensitive to visibility and surface appearance, but no fixed slow-loop architecture follows.

Contact modeling can be a substantial source of sim-to-real mismatch. Contact geometry, friction, compliance, and numerical handling of impact can differ from physical behavior, and their effects interact with actuation and sensing. This chapter does not measure whether contact is the largest gap for any particular robot; that comparison requires target-system tests.

We can state one transfer objective as follows: train a fixed policy $\pi_\phi$ on a source distribution $p_\text{src}(\theta)$ over environment parameters and evaluate its return $J(\pi_\phi;\theta)$ under a target distribution $p_\text{tgt}(\theta)$. We want $\mathbb{E}_{\theta\sim p_\text{tgt}}[J(\pi_\phi;\theta)]$ to remain useful. The difference between source and target expectations depends on both the distribution shift and how the fixed policy's return varies with $\theta$; coverage alone does not establish target performance.

We can see the coverage argument directly. On a single environment-parameter axis, a narrow source concentrates its mass far from the target, while a wide source spreads its probability mass across a broader range that includes the target.

One technique is domain randomization: train on varied simulated dynamics and terrain so the policy encounters more of the conditions it may face later. Coverage of a target parameter range does not guarantee small transfer error, and finite data, mismatched observation and action interfaces, and optimization can still matter. Curriculum-style schedules may begin with a narrower range and widen it later, but their effect must be measured for the particular task and robot.

A second technique is system identification: use real observations to estimate parameters relevant to prediction or control, then adapt a simulator, model, or policy accordingly. Estimation may be performed offline or online. The architecture and measured benefit depend on the robot; this chapter does not establish a general actuator-calibration and terrain-adaptation pattern for Cheetah-style controllers.

A third is residual or adaptation learning: start from a model or policy trained under nominal conditions and learn a correction from target-domain data. The amount of real data and risk during adaptation depend on coverage, action limits, and validation. A nominally working policy does not by itself make online adaptation safe, and no broad industrial prevalence claim follows from this example.

A fourth is rapid adaptation: train a policy or estimator across varied conditions so it can use a short recent history, context vector, or a few update steps on a new condition. Terrain and payload changes can make this useful for locomotion, as demonstrated in named adaptation studies. Whether rapid adaptation outperforms a well-tuned fixed or robust controller depends on the target distribution and evaluation.

One approach to cross-embodiment modeling is to represent the robot as a graph of links and joints. Shared update rules can operate over different graph structures, while body-specific encoders, decoders, or action interfaces handle differing sensors and actuators. This is a design pattern, not a guarantee that a single learned model transfers across arbitrary morphologies.

The body-graph approach addresses one practical interface problem. Flat joint-angle vectors have different lengths for robots with different joint counts, so a fixed-width model needs padding, adapters, or another alignment scheme. A graph model instead shares a local update rule across links and joints of varying number. That gives it a mechanism for parameter sharing across body sizes, but generalization still depends on training coverage and physical similarity.

Training across many bodies could help a model learn reusable structure, but it also expands the data and validation problem. A systematic error in shared parameters can affect multiple morphologies, and aggregate performance can hide body-specific failures. Evaluation must therefore report results by body and task rather than only a pooled score.

As in Part X, Ch 2: Robotic Manipulation, transfer depends on which physical and sensing structure is shared. Locomotion changes foot geometry, gait, and contact forces; manipulation changes end effectors, objects, and grasp contacts. Neither domain is uniformly easier to transfer across, and methods from one require validation before reuse in the other.

We can recast the whole discussion as a world-model transfer problem. Suppose we have a source world model $f_\text{src}(s, a)$ trained in simulation, and we want a target model $f_\text{tgt}(s, a)$ that predicts the physical robot. The simplest transfer model assumes that the target dynamics differ from the source dynamics by an additive residual, which lets us write:

where:

• $f_\text{src}(s, a)$: the source world model, trained in simulation
• $f_\text{tgt}(s, a)$: the target world model, describing the physical robot
• $\Delta(s, a)$: the residual dynamics not captured by the source model
• $s$: the robot state (e.g., joint angles and velocities)
• $a$: the control action

The additive form is a modeling assumption, not a law. It requires comparable state and action interfaces; even then, a learned residual may not extrapolate reliably across changes in friction, mass, or compliance. When robots have different state or action dimensions, a direct additive residual is undefined until their interfaces are aligned. Padding, adapters, shared task-space variables, or body-graph representations are possible alignments; a graph is not required.

These four transfer techniques address different parts of the transfer problem; not all require the additive residual model:

• Domain randomization varies simulated parameters to expose a policy or model to a range of conditions. It does not require the residual $\Delta$ to have zero mean; its benefit must be measured on target conditions.
• System identification estimates physical dynamics parameters from real rollouts; a residual model may instead learn parameters of $\Delta$ directly.
• Residual learning trains a neural network to approximate $\Delta$ directly.
• Meta-learning trains a policy whose behavior depends on a context that is inferred from a few real rollouts.

These techniques can be combined. Domain randomization exposes the learner to anticipated variation, while residual learning or online estimation can correct systematic target mismatch observed on hardware. Neither contribution is automatic; the combination must be evaluated on the target robot and terrain.

We close with a small experiment that makes these ideas concrete. We generate two training sets: one uses a single fixed terrain, while the other uses ten randomly drawn terrains with three episodes on each. We fit a linear model of the vertical hopper dynamics on each set, then evaluate on a held-out terrain absent from both training sets.

In this seeded toy comparison, the wide-trained predictor has lower mean open-loop height error on the single held-out terrain. That is not a closed-loop control result or evidence that widening the source always helps. A simple distributional bound explains one consideration: if the fixed policy's return $J(\pi;\theta)$ on an environment parameter $\theta$ lies between zero and $J_{\max}$, then

where:

• $J(\pi;\theta)$: return of the fixed policy $\pi$ in environment $\theta$.
• $p_\text{src}$ and $p_\text{tgt}$: source and target distributions over the same environment-parameter space.
• $J_{\max}$: an assumed finite upper bound on nonnegative return.
• $D_{\mathrm{TV}}$: total variation distance, defined as $\sup_A|p_\text{src}(A)-p_\text{tgt}(A)|$ over measurable sets $A$.

This bound is loose and says nothing about whether a learned policy succeeds on the target. A wider source can reduce some distribution mismatch while also making learning harder or placing too little training mass on important target conditions. Measure target performance directly and distinguish open-loop model error from closed-loop return.

The modeling and planning methods in this chapter are useful in legged and mobile robotics, but they carry limitations that affect system design and evaluation.

The first limitation is that contact-mode changes are difficult for a single smooth predictor to represent accurately. A model fitted across a switch can blur the event or mistime it, as the hopper experiment showed. Contact schedules, event predictors, and discrete mode variables make the switch explicit; contact-implicit optimization instead keeps the contact decision inside a constrained numerical problem. Each choice trades modeling detail against computational and estimation demands.

The second limitation is that one-step prediction quality need not imply good decisions. Error can accumulate under rollout and can be concentrated near task-critical contact events. A limited bound shows what can be concluded. For one fixed open-loop action sequence $a_{0:H-1}$, suppose each stage cost and the terminal cost are $L$-Lipschitz in state, the same actions are evaluated by both models, and both trajectories start from the same state. Then:

Here $\hat{s}_t$ and $s_t$ are the modeled and true states reached under that same action sequence. The bound follows by applying the Lipschitz condition to each cost term and then the triangle inequality. It does not compare two different policies. To bound the regret from choosing actions with the model, one additionally needs a uniform bound $\left|C_{\text{model}}(a)-C_{\text{true}}(a)\right|\leq\delta$ for every candidate sequence in a common feasible set. If $a_{\text{model}}$ minimizes modeled cost and $a_{\text{true}}$ minimizes true cost over that set, then $0\leq C_{\text{true}}(a_{\text{model}})-C_{\text{true}}(a_{\text{true}})\leq 2\delta$. A small error on one observed trajectory alone does not supply that uniform guarantee.

The evaluation question "does this world model help you act?" is the one that matters for locomotion, and it is not answered by looking at one-step prediction loss. We return to this in depth in Part XI, Ch 4: Planning, Control, and Policy Evaluation.

The third limitation is model exploitation in optimization. An MPC optimizer can prefer a plan that relies on contact forces the physical robot cannot generate if its model or constraints permit them. Friction, actuator, and balance constraints can rule out some such plans when their parameters and contact assumptions are valid, but the constraints can also be wrong. Structured prior knowledge helps narrow the search; it does not by itself establish hardware safety. Identification, monitoring, and closed-loop tests remain necessary.

The fourth limitation is sim-to-real mismatch. A new body, payload, sensor, or terrain can change which simulator errors matter. Domain randomization, system identification, and residual learning have reduced specific gaps in published systems, but their effects are not automatic or interchangeable. Deployment-specific characterization and fallback behavior may be useful; this chapter does not establish how frequently companies use a particular fallback design.

The fifth limitation is data coverage. Physical-robot trajectories can be costly and valuable for validating real contact and sensing, while simulation can provide more controlled variation at lower collection cost. Required data volume and the best data source depend on the model and task. Pretrained approaches discussed in Part IX: Foundation and World-Action Models may reduce per-deployment data in some settings, but cross-embodiment locomotion transfer still needs measured evidence.

The sixth limitation is that navigation can require distinct estimates of metric layout, semantic content, and moving agents. A system may represent them separately or jointly. If dynamic obstacles or doors are not updated, a route planned from an older map can become invalid. Testing map updates and distinguishing persistent from changing structure matter when people or other agents share the space.

These methods are relevant to deployed robots, but deployment evidence should be named and measured rather than inferred from the toy experiments. Boston Dynamics documents Spot use on construction sites, and warehouse operators document mobile-robot fleets. This chapter does not quantify fleet-wide daily floor area, establish a single dominant humanoid deployment pattern, or attribute every deployment's success to one world-model family. Explicit mapping, model-based control, and learned dynamics are different design choices whose contribution must be evaluated in each system.

Locomotion, humanoids, and navigation are the application domain where world models meet the physical body. We have seen that:

• Body dynamics and terrain interaction can involve hybrid contact modes, uncertain surfaces, and an underactuated floating base. The 1D hopper illustrates mode switching, but not every legged controller needs a latent terrain estimate or a flight mode.
• Whole-body control can connect a centroidal plan to joint torques under contact and actuator constraints. Model-based MPC predicts and replans; a direct learned policy need not follow that loop. The useful horizon and update rate are task-dependent.
• Spatial memory can support long-horizon navigation. Metric, topological, semantic, and learned maps expose different information; a map can be a probabilistic belief or a deterministic representation. Their relative value needs task-specific evaluation.
• Cross-embodiment and sim-to-real transfer require compatible interfaces and target-domain tests. Domain randomization, system identification, residual learning, and rapid adaptation address different mismatches; they do not all assume one additive dynamics residual.

The through-line is that a locomotion or navigation world model is never just $p(s_{t+1} \mid s_t, a_t)$. It is a structured composition of body, terrain, and task, each with its own uncertainty and its own failure modes, and each requiring its own evaluation. The next chapter turns to autonomous driving, where the map is a road network, the body is a car, and the road surface and other agents pose distinct prediction problems.

Ready to test your understanding? Take this quick quiz to reinforce what you've learned about locomotion, humanoids, and navigation world models.