Locomotion, Humanoids, and Navigation › Spatial Memory and Goal Navigation
World Models HandbookAll chapters Locomotion, Humanoids, and Navigation › Spatial Memory and Goal NavigationOne section · Sections in this chapter4 / 8 ↓
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
Locomotion determines how the robot moves its body. Navigation determines where it should go. The two are coupled: the choice of gait depends on the destination and obstacles, and the choice of route depends on what gaits the terrain permits. In an autonomous mobile robot, the coupling is managed through a world model that includes a map: an internal representation of the environment that supports spatial reasoning over long horizons.
The map is not a luxury. A robot's sensors have finite range; a decision about where to go next must often be made based on information that left the field of view minutes ago. The environment is partially observable, and the belief state includes the robot's estimate of layout, occupancy, and semantics of the space. This is the navigation-specific instance of the POMDP belief state from Part II, Ch 1: Markov and Partially Observable Decision Processes and Part II, Ch 2: Bayesian Filtering and Belief States. In the navigation case the latent state includes the robot pose , the map , and often the motions of other agents, while the observations are range or image measurements and the actions are velocity commands or waypoints.
One reason to build a map is to support choices beyond the current sensor view. A purely current-observation policy cannot distinguish two locally identical hallways with different known destinations; stored spatial information can. An explicit map is one such memory structure, while a recurrent policy or other memory can also retain useful history. A probabilistic map may be part of the robot's belief over persistent layout, but maps need not always be represented as posteriors.
What a Navigation World Model Contains
Several map representations appear in navigation. A metric occupancy grid or volumetric map assigns occupancy estimates to spatial cells, as in OctoMap. It can support local planning, but dense high-resolution coverage of a large environment can be costly. Sparse, hierarchical, feature-based, and graph representations have different storage and query scaling; SLAM does not imply a dense grid.
The second layer is topological: a graph whose nodes are landmarks or rooms and whose edges represent traversability, as in the classical work on topological SLAM and in Chaplot et al. (2020), "Neural Topological SLAM for Visual Navigation". Topological maps are scale-efficient and support long-horizon reasoning, but they require a mechanism to decide which observations become nodes. That mechanism is the crux: a topological map is only as good as its node placement, and getting the granularity right, coarse enough to be compact and fine enough to be informative, is a real design problem.
The third layer is semantic: object or place categories and their relations. Such information can provide priors for where a goal object might be, including when geometry is incomplete. Vision-language models may supply candidate semantic priors, but they cost computation and can be wrong; a category association is not an observation of what lies behind a closed door.
Different systems combine these representations according to task and scale. A metric map can help resolve nearby obstacles; a topological graph can organize long routes; semantic labels can guide object search. None has a single exclusive function, and this chapter does not establish a universal most-robust combination. The useful mixture depends on localization quality, map updates, route length, and the cost of mistakes.
SLAM as a World-Model Build
Simultaneous localization and mapping (SLAM) is a family of methods for jointly estimating a robot trajectory and a map from sensor data. One probabilistic formulation tracks a belief over current pose and map :
where:
- : robot pose at time (position and orientation)
- : map of the environment (e.g., occupancy grid, landmarks)
- : sequence of observations from time 1 to
- : controls applied between successive poses through
- : belief over pose and map given all observations and controls
The observations through time and controls applied before time condition a joint belief over the robot's location and the surrounding world. This is a world model that is updated from data and queried by a downstream planner. The chicken-and-egg structure is why SLAM is hard: the robot needs its location to build a consistent map, and it needs a map to estimate its location. Joint inference addresses both unknowns together. Other indexing conventions are possible if the time interval attached to each is defined consistently.
SLAM systems make different structural choices. EKF-SLAM maintains a Gaussian approximation over pose and landmarks, with covariance updates that can be expensive as landmark count grows. Factor-graph and bundle-adjustment systems, including Cartographer and ORB-SLAM, optimize pose and measurement constraints. Their speed and loop-closure reliability depend on data association, graph size, solver, and implementation; graph structure alone does not guarantee either advantage.
Loop closures are one payoff of the graph formulation. Dead reckoning can let pose uncertainty accumulate along a trajectory; a valid, informative loop-closure constraint can reduce drift or uncertainty after optimization. A mistaken association may instead damage the estimate.
## Synthetic pose-variance profiles along a trajectory.
## Without loop closure the variance grows linearly with pose index; a
## stylized loop-closure effect begins after pose 80 and variance then decays.
n_slam = 120
slam_t = np.arange(n_slam)
cov_no_loop = 0.02 * (slam_t + 1)
cov_with_loop = cov_no_loop.copy()
for i in range(80, n_slam):
cov_with_loop[i] = cov_with_loop[i] * np.exp(-0.06 * (i - 80))
Neural implicit mapping can represent geometry with fields queried at spatial points; some systems combine such fields with explicit structures. Neither an explicit nor a learned map is automatically compact, cheap to query, complete, or calibrated in unobserved regions. Explicit maps can retain unknown cells or uncertainty, while learned maps may interpolate plausible geometry that needs validation. Their value for planning must be measured on the target scene and query workload.
Learning to Navigate from a Built Map
A map supports goal-directed navigation, but localization, perception updates, and motion control remain necessary. Three illustrative planning strategies follow.
The first is geometric planning: run A*, D*, or a sampling-based planner over an occupancy grid or topological graph, then follow the resulting path. This classical approach remains useful where the map and travel constraints can be represented explicitly. The map is the spatial model, the planner computes a policy or route, and the objective may be path length or a risk-adjusted cost. Its explicit representation often makes a bad route easier to diagnose: one can inspect the map cells or graph edges that the planner used.
The second is learned navigation with a spatial map. Chaplot et al. (2020), "Learning to Explore using Active Neural SLAM," combine a learned map-and-pose module, hierarchical policies, and analytical path planning. They reported stronger exploration performance and sample efficiency than the particular end-to-end baselines they evaluated. That result supports explicit spatial structure as a useful inductive bias in their setting; it does not establish a universal advantage over recurrent or other map-free policies.
The third strategy uses learned spatial features. A network may write observation-derived features into cells or graph nodes and let a policy or planner query them. This extends temporal memory with spatial indexing (Part III, Ch 4: Temporal State and Memory). The cells may be deterministic latent features or an explicit uncertainty-aware belief; latent storage alone does not require Bayesian posterior planning or guarantee accurate geometry.
These strategies vary in how much geometry and uncertainty they expose. Explicit maps can make a route easier to inspect but can contain unnoticed estimation errors; learned features can help with ambiguous observations but may also hallucinate missing geometry. Storage, query cost, calibration, and transfer vary by implementation. A hybrid map is one option, not an established most-robust design for every navigation task.
Why Navigation World Models Are Different From Locomotion Ones
Compared with a local gait controller, a navigation model often emphasizes two additional demands; a full locomotion system can also face them.
First, navigation can span far more time and distance than one gait cycle. Long-route planning therefore benefits from abstract state such as occupancy grids or topological maps. A fine-grained dynamics rollout over a long route would require many model steps and incur substantial computation and potential compounding error; a route planner can instead work at a coarser scale and leave local motion to the locomotion controller.
Second, some navigation actions gather information rather than shorten the immediate route. Turning a corner may reveal whether a hallway continues; that observation can improve a later route choice. Part VII, Ch 5: Active Perception, Dual Control, and Exploration develops this trade-off. Exploration is useful when the expected value of a better map outweighs its travel and sensing costs; it is not automatically the best choice on every task.
We now make the geometric planner concrete by building an occupancy grid, running value iteration on it, and extracting a path.
## Build a simple occupancy grid for a house-like environment
B = 25
occupancy = np.zeros((B, B), dtype=bool)
## Walls and obstacles (rows, cols)
occupancy[5:20, 12] = True # vertical wall with a gap
occupancy[5, 5:12] = True # horizontal wall
occupancy[15:20, 8] = True # small block
occupancy[15:20, 18] = True # small block
occupancy[10:15, 18:22] = True # table in another room
start = (2, 2)
goal = (22, 22)
occupancy[start] = False
occupancy[goal] = False
## Value iteration: V(state) = -1 + max_a gamma * V(next state)
gamma = 0.95
V = np.zeros_like(occupancy, dtype=float)
actions = [(-1, 0), (1, 0), (0, -1), (0, 1)]
for _ in range(400):
V_next = V.copy()
max_delta = 0.0
for r in range(B):
for c in range(B):
if occupancy[r, c] or (r, c) == goal:
continue
best = -np.inf
for dr, dc in actions:
nr, nc = r + dr, c + dc
if 0 <= nr < B and 0 <= nc < B and not occupancy[nr, nc]:
best = max(best, gamma * V[nr, nc])
if best > -np.inf:
V_next[r, c] = -1.0 + best
max_delta = max(max_delta, abs(V_next[r, c] - V[r, c]))
V = V_next
if max_delta < 1e-4:
breakThe value iteration loop is worth reading as a compressed statement of what planning on a map means. Each sweep visits every free cell, looks at its four neighbors, computes the best discounted neighbor value, and charges a step cost. Repeated sweeps propagate information backward from the goal until values stop changing. In this deterministic grid with a zero-valued terminal goal and constant step penalty, the converged value is the negative discounted cost along a best route. Selecting the neighbor with the highest value yields a shortest feasible path here. The map supplied the geometry; the iteration supplied the policy.
We then extract a greedy path from the start to the goal.
## Greedy path extraction from the value function
path = [start]
current = start
visited_steps = 0
while current != goal and visited_steps < 200:
best_val = -np.inf
best_next = None
r, c = current
for dr, dc in actions:
nr, nc = r + dr, c + dc
if 0 <= nr < B and 0 <= nc < B and not occupancy[nr, nc]:
if V[nr, nc] > best_val:
best_val = V[nr, nc]
best_next = (nr, nc)
if best_next is None:
break
path.append(best_next)
current = best_next
visited_steps += 1
path = np.array(path)
print(f"Path length: {len(path)} states ({len(path) - 1} moves)")
print(f"Final state: {path[-1]}")The value function plot and the path make clear what the map-based world model gives us.
The path is not a straight line to the goal because the world model encodes the geometry of the environment. It follows open column 2 to row 22, then traverses row 22 to the goal, avoiding the interior walls and blocks. Value iteration is the classical answer, and it is instructive to note what is not happening: you are not learning a policy, you are computing a policy from a known map. The value iteration update is:
where:
- : the discounted return from under the best route in this deterministic grid, with reward per step and zero terminal value.
- : the discount factor, with , weighting future costs relative to immediate ones.
- : a neighboring state reached by taking action from state .
- : the per-step cost, a penalty charged for each step taken.
The max is over admissible neighboring states that are not occupied. The same value-iteration update can use occupancy estimates from a learned mapper if states and transitions are defined for them. Other learned-map systems use graph search, analytical planners, or policies instead; Chaplot et al.'s system is not this notebook's value-iteration loop. Here the map supplies spatial state and value iteration computes a route. A body MPC and a route planner can both be updated as observations change, but their models, actions, and replanning schedules differ.