Robustness, Calibration, and Safe Control › Summary

• Input shift changes the input distribution; dynamics shift changes the transition law and may also change subsequently visited inputs. Observation and task changes can co-occur with either, while hidden parameters provide one mechanism of dynamics change.
• Stress testing combines falsification and measurement. Specify plausible interventions, measure their consequences, and record what remains untested. Passing a finite suite does not establish safety outside it.
• Ensemble disagreement is a diagnostic, not a detector guarantee. Our input-shift population produced a larger disagreement increase than the damping-shift population. Aggregate one-step error alone concealed some of the population difference.
• Calibration means coverage matches its specified claim. Reliability diagrams measure empirical marginal coverage at displayed levels; report interval width too. Global rescaling may help a new population but does not automatically transfer. Conformal guarantees need exchangeability and do not imply conditional or trajectory safety.
• Abstention needs an informative ranking and a defined alternative. Risk-coverage curves evaluate retained predictions, not the safety of rejected cases. Neither percentile nor absolute thresholds universally survive shift.
• For integrable costs, CVaR emphasizes a selected upper tail and approaches the mean as its level decreases to zero. Near one it emphasizes extremes. Model mismatch and finite tail samples can prevent realized improvements; a soft penalty is not a hard safety constraint.
• Runtime assurance splits the controller. A performance controller does the task; a certified fallback maintains the invariant; a monitor decides who is in charge and must trigger before the state leaves the recoverable set.
• Separate diagnosis from certification. Innovation checks are reactive diagnostics. A prospective reachable-set check carries a guarantee only with sound enclosures, disturbance and timing assumptions, and a safe continuation. Our scenario-based shield demonstrates switching, not that proof.

The next chapter, Security, Ethics, Privacy, and Governance, extends the scope to adversarial, societal, and institutional concerns. The distinction developed here remains useful: a diagnostic can inform a decision without proving that the decision satisfies an engineering or governance requirement.