Autonomous Driving › Autonomous Driving

A van is double-parked on the near corner of a four-way intersection. From the ego vehicle's perspective, the van blocks the view down the cross street. The ego is thirty meters out, moving at forty kilometers per hour. Somewhere beyond the van, an unseen vehicle may be rolling toward the same intersection, waiting to yield, or absent. In less than three seconds at constant speed, the ego would reach the crossing; its safe response depends on what is hidden and on its available stopping distance.

Occluded intersections are a recognized challenge in autonomous driving. The scene illustrates a central problem: acting under partial observation. The van may itself obstruct travel. Uncertainty about cross traffic adds a planning hazard. A driver might slow until the view improves; an autonomous system has to handle the uncertainty computationally. A world model offers one way to organize the estimate and its consequences. Understanding when that model deserves to be trusted is what this chapter is about.

A driving world model used for planning should support action-relevant estimates of spatial state, hidden hazards, other-agent motion, and uncertainty, whether those quantities are explicit outputs or implicit in its representation. Depending on the system, some of this work may sit in separate perception and prediction modules rather than one model. A convincing rendered video does not establish that the predicted state supports a safe braking decision. The important test is what actions the system takes when its state estimate is wrong or incomplete.

That distinction organizes this chapter. In earlier chapters we developed world models as a general idea, then layered in representations, dynamics, and planning. Here we ask what changes when the world is a road: what state matters, why predicting other agents is especially difficult, why closed-loop simulation differs from open-loop prediction, and how rare events, safety, and planning interact. Each question asks what the model must get right to support a decision. A model trained for generative fidelity and one trained for decisions can share an architecture yet differ substantially in what each represents well.

Autonomous driving is an application where observability, controllability, and safety meet latent dynamics and multimodal prediction under partial observation. Observability concerns what state can be inferred from observation and action history under assumed dynamics; controllability concerns which states the ego can reach through its actions; safety concerns which outcomes are acceptable. A learned model can represent the current scene and possible evolutions, but it does not remove the limits imposed by hidden state and uncertain behavior. Part IV develops the spatial and physical foundations.

We will work through four components: BEV and occupancy state, other-agent behavior prediction, closed-loop driving simulation, and rare-event safety planning. They interact: a prediction can be compromised by a poor spatial estimate, and a planner that relies only on predicted candidate modes cannot evaluate a hazardous mode the predictor omitted. We close with a CPU-sized two-agent intersection experiment comparing a constant-velocity baseline, a single learned trajectory, a label-supervised two-mode output, and an oracle. That toy measures one decision and its simulated outcome; it is not a full closed-loop driving benchmark.