Robustness, Calibration, and Safe Control › Risk-Sensitive and Constrained Planning

Everything so far has been about the model. Now we change the objective. Minimizing expected cost is risk-neutral with respect to that cost: a certain cost of 10 and an equally likely cost of 0 or 20 have the same expectation. A point prediction does not force risk neutrality; a controller can still use conservative margins, robust constraints, or nonlinear costs. A predictive distribution permits explicit tail objectives, whose suitability depends on the application's losses and requirements.

Two families of alternatives recur in practice:

• Risk-sensitive objectives use a tail-sensitive functional. For integrable costs, CVaR at level $\alpha$ averages the worst $(1-\alpha)$ probability mass, using fractional boundary mass when necessary. It approaches expected cost as $\alpha$ decreases to zero; near one it emphasizes increasingly extreme upper-tail costs. Here $\alpha=0.75$ selects the worst quarter.
• Constrained objectives separate feasibility requirements from performance. A pointwise constraint limits predicted states; a chance constraint such as $\Pr[|p|>p_{\max}]\le\delta$ imposes a hard bound on violation probability, not a promise that every realized state stays inside. The objective can itself be expected cost or a risk-sensitive functional.

The expected-cost objective summarizes average loss; CVaR summarizes a chosen upper tail. These answer different questions. Expected cost can penalize rare failures heavily if their assigned loss is large, and a hard constraint can reject them without CVaR. Conversely, minimizing CVaR can still permit a safety violation if it improves the chosen tail objective. Specify safety requirements separately from performance preferences.

A risk-sensitive planner needs a distribution of outcomes for a candidate action sequence, not a single rollout. Several sources are available, and they are not equivalent:

• Ensemble members as samples. Run the candidate sequence through each member. This uses variation between fitted models as an approximation to epistemic uncertainty, i.e., uncertainty about the model fit. Its quality depends on both coverage of plausible dynamics and the weights assigned to members; bootstrap ensembles do not by themselves guarantee calibrated tail probabilities. See Part VIII: Decision-Centric Research Lineages for the lineage.
• Residual-noise propagation. Add sampled process noise to each member's predicted transition. In this toy plant, position and velocity noise innovations satisfy $\epsilon_t^{(p)}=\Delta t\,\epsilon_t^{(v)}$, so they must not be sampled independently. This relation concerns the noise contributions, not the full state increments. Combining model fits with this noise defines an approximate rollout distribution; one-step calibration does not validate its multi-step joint distribution.
• Scenario evaluation. Evaluate explicit parameter and disturbance choices. A finite scenario set is inspectable, but checking it does not prove safety for unsampled conditions. Scenario-optimization guarantees require additional sampling, optimization, and support assumptions; merely naming a chosen schedule worst case supplies none.

The code combines alternative fitted models with residual-noise draws. It does not explicitly add total predictive variance on top of ensemble variation. Nevertheless, held-out residual variance can itself include fitting error, so this construction is not a decomposition into independent uncertainty sources. Its joint rollout spread requires validation; over- or under-dispersion and missed model families can affect planning.

The running environment already includes Gaussian process noise. We now add a soft envelope penalty to stage cost when $|p|$ exceeds a chosen threshold. This can amplify expensive excursions, but it is not a control barrier function or a hard constraint. Quadratic costs can also have substantial tails, and neither this penalty nor CVaR guarantees different actions or improved realized tail costs. The experiment measures those outcomes rather than assuming them.

The distinction between averaging across all outcomes and averaging across the worst outcomes is easiest to see directly on a cost distribution. The cell below draws a right-skewed set of synthetic episode costs and computes both summaries, using the same sample estimator the planner optimizes.

At the same state and RNG state, both modes generate the same candidate actions, member assignments, and linked noise draws. Only their cost aggregation changes: mean versus empirical CVaR. Once actions differ, closed-loop states and therefore imagined futures differ too. Using the same seeds pairs the search randomness; it does not make later state-conditioned predictions identical.

We pair environment noise streams and search seeds across controllers. Different actions still lead to different states. Common random numbers reduce the variance of a difference when paired outcomes have positive covariance: $\mathrm{Var}(A-B)=\mathrm{Var}(A)+\mathrm{Var}(B)-2\mathrm{Cov}(A,B)$. They can increase it when covariance is negative. This experiment reports a paired finite comparison, not a measured reduction in required episode count; see Glasserman and Yao's conditions for common random numbers.

A finite soft penalty permits a tradeoff: a violating candidate can win if reductions in other cost terms outweigh its extra penalty. Our zero-target position cost does not make $|p|=0.7$ intrinsically better than $|p|=0.55$. Nor does this example establish that no penalty choice could enforce feasibility in any problem: some bounded or finite problems admit exact penalties. The quadratic penalty here supplies no proof of pointwise safety. If an envelope is a requirement, encode and justify it separately rather than treating a tuned weight as a certificate.

The alternatives each have costs:

• Hard constraints on a nominal rollout enforce $|p|\le p_{\max}$ at modeled steps. Feasibility establishes that modeled property, not automatically the real plant's safety. Connecting the two requires justified error and disturbance assumptions.
• Chance constraints enforce $\Pr[|p|>p_{\max}]\le\delta$ under a specified probability law. Computing or estimating that probability can be difficult. Finite samples can under- or overestimate it, and rare violations can be absent from the sample. A bound on violation probability is a different requirement from all-disturbance invariance.
• Runtime assurance. Check a proposal against an independently justified safety condition and select a verified fallback when necessary. Guarantees require a sound decision module, an appropriate initial condition, and a fallback with proven properties under the declared plant and disturbance assumptions. Our next experiment illustrates the switching architecture without satisfying those proof obligations.

The code above implements a soft penalty and sampled CVaR, not constrained optimization. Zero observed violations among eighteen scenarios does not imply zero violation probability. A predictive safety filter needs justified uncertainty bounds and recursive feasibility; see Wabersich and Zeilinger. For continuous control-affine dynamics $\dot s=f(s)+g(s)a$, a continuously differentiable barrier $h$ describes $h(s)\ge0$ and constrains actions through $\dot h(s,a)+\gamma(h(s))\ge0$, with an appropriate extended class-K function $\gamma$ (for example $\gamma(r)=\lambda r$, $\lambda>0$). Forward invariance additionally needs regularity, feasible actions, correct dynamics or robust error bounds, an admissible initial state, and satisfaction over time. Sampling and delay require additional treatment. Our quadratic penalty is not this condition; see Ames et al..