Robustness, Calibration, and Safe Control › Robustness, Calibration, and Safe Control
World Models HandbookAll chapters Robustness, Calibration, and Safe Control › Robustness, Calibration, and Safe ControlOne section · Sections in this chapter1 / 8 ↓
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
A hypothetical warehouse robot learns a world model on a dry floor. Your planner trusts narrow prediction intervals when choosing a maneuver. If the floor becomes wet, the relationship between action and motion can change without the model's confidence changing immediately. You may then choose an action that was reasonable under the old dynamics but inappropriate under the new ones. This is a motivating scenario, not a reported robot experiment; the executable example below uses a synthetic oscillator not a wheel-torque model.
Accuracy on the training regime does not establish accuracy after this change. What failed is the assumption that the old transition law and uncertainty estimates still apply. This chapter examines how to test those assumptions and how to limit reliance on them.
The preceding chapter, Failure Modes and Model Exploitation, concerns how world models break. Here we develop an engineering response through four connected pieces:
- Distribution shift and stress testing: how to evaluate specified changes before deployment and monitor observed mismatches.
- Calibrated uncertainty and abstention: how to make the model's own confidence a usable decision signal, and how to decline to act when it is not.
- Risk-sensitive and constrained planning: how to plan against a distribution of futures rather than a single imagined one.
- Fallback policies and runtime assurance: how to separate a performance controller from a fallback, and what must be proved before calling that fallback safe.
These topics interact. Stress testing identifies specified changes worth evaluating; calibration measures whether predictive probabilities agree with outcomes in an evaluated population. Neither is a universal shift detector. Risk-sensitive planning changes how sampled futures are scored, while runtime assurance checks whether the performance controller may continue operating under independently justified assumptions. Each link needs evidence: a calibration estimate does not prove future coverage, a risk objective does not enforce a hard constraint, and a fallback is not certified merely because its test trajectories stay in bounds.
A world model is a predictor. A controller is a decision rule. Good one-step prediction does not establish safe closed-loop behavior. A fallback can reduce dependence on a weak model, but safety requires verified assumptions about the plant, disturbances, state estimates, timing, switching logic, and fallback's operating region. The toy fallback in this chapter has no such certification.
Throughout, we use a one-dimensional positioning task with a damped oscillator as the plant. The printed tables come from executable code, so you can change the assumptions and observe the consequences. This modest example exposes the distinction between prediction accuracy, uncertainty estimates, and closed-loop outcomes. Larger learned systems introduce additional representation and estimation questions; scale alone does not establish that confidence tracks competence.