Autonomous Driving › Rare Events, Safety, and Planning

Lane-keeping, vehicle-following, and intersection negotiation are recurring driving tasks, but rare hazardous situations deserve separate attention. A child entering the road unexpectedly or a cyclist swerving around a parked truck may be infrequent in a dataset while still mattering greatly to safety. Even high average prediction accuracy can conceal failures concentrated in such cases. The evaluation must therefore report more than an aggregate error rate.

This has several implications for how driving world models are built and evaluated.

Rare-event coverage. Limited or unrepresentative training examples can make a model underpredict a hazardous behavior, such as wrong-way driving; absence from the training set does not prove the model cannot generalize to it. Responses include collecting and mining more data, synthesizing counterfactual cases while checking their fidelity, and adding uncertainty-aware monitors or fallbacks. These approaches can be layered, but each needs its own evaluation: more examples do not guarantee coverage, synthetic events may differ from real ones, and an uncertainty signal does not by itself specify a safe maneuver.

Calibrated uncertainty. A planner needs to know what a model's probabilities mean for the event it uses to decide. For a stated class of cases, a calibrated 90% prediction interval should contain the outcome about 90% of the time; a calibrated 10% collision-risk estimate should correspond to collisions in about 10% of comparable cases under the specified action and horizon. Those are different calibration questions. A risk threshold is meaningful only if the collision event, operating conditions, and model probability are defined consistently. Inflated collision-risk estimates for safe cases can cause avoidable interventions under a conservative policy; underestimated risk can conceal danger.

Calibration is hard to establish for multimodal predictions. Temperature scaling can recalibrate suitable logits. Conformal methods can provide marginal coverage under their stated assumptions. Ensembles can estimate disagreement. Shift detectors can flag departures from training conditions. These methods target different quantities, and none alone supplies calibrated collision risk in every deployment condition. As we will see in Robustness, Calibration, and Safe Control, calibration measured on one distribution does not automatically hold after distribution shift.

Calibration measured only on logged ego actions also does not validate collision-risk probabilities for alternative actions. Those counterfactual estimates need the action-support and causal-response assumptions discussed above, or suitable interventional validation.

A planner that consumes a multimodal prediction has to convert the distribution into an action. Several strategies exist:

• Worst-case. Choose an action that minimizes the maximum modeled cost over represented modes; the worst mode can differ by action. This is conservative relative to those futures, but not a guarantee against omitted hazards.
• Expected-cost. Weight modes by their probabilities and minimize expected cost. This criterion can still select an action with a low-probability severe outcome when its modeled expected cost is lower; it does not separately bound tail risk. Computational cost depends on the model and optimizer.
• Chance-constrained. Require the modeled probability of collision to remain below a chosen threshold. Its reliability depends on the probability model and how the collision event is defined.
• Distributionally robust. Optimize against the worst-case distribution within an ambiguity set. This can hedge distributional misspecification when the set covers plausible models, sometimes producing more conservative actions.

These strategies differ in how they treat rare costly outcomes. Worst-case planning considers the most adverse modeled case; expected-cost planning weights cases by modeled probability and consequence; chance constraints set an upper bound on modeled violation probability; distributionally robust planning considers a set of plausible probability models. None is automatically safe when relevant hazards are missing from the model. The tradeoff between progress and conservatism depends on the operating design domain and safety requirements.

Model exploitation. A planner optimized against an imperfect learned model can favor actions where that model is overly optimistic, as discussed in Failure Modes and Model Exploitation. For example, underestimating red-light violations can make an intersection crossing look safer than it is. The planner may be following its objective correctly while acting on an incorrect risk estimate.

Ensemble disagreement, targeted stress tests, runtime monitors, and independently specified constraints can expose or limit some model errors. A fallback or minimum-risk maneuver may be part of a system design, but a confidence threshold alone cannot certify that the learned model recognizes every unsafe case. The fallback must itself be tested against sensing, dynamics, and road-rule failures.

The Role of Planning. The planner selects ego actions against predicted futures while respecting road rules, physical limits, and uncertainty. Sampling-based MPC, discussed in Sampling-Based Planning and Model Predictive Control, is one approach: propose candidate action sequences, roll them out through a model, score them, and execute a short prefix before replanning. Better predictions can improve this process, but safety also depends on the objective, constraints, fallback behavior, and errors outside the learned model.

The coupling between world model and planner is the central theme of this chapter. In an application like autonomous driving, the world model is not a neutral representation. It is a decision-making instrument. Prediction metrics and action-outcome tests answer different questions; neither alone establishes real-road safety. A useful evaluation connects errors in the model's output to decisions made with that output.