Sampling-Based Planning and Model Predictive Control › Summary

Sampling-based planning and MPC are the workhorses that make a black-box world model useful for control. The key ideas:

• Random shooting samples $N$ action sequences, rolls each through the model, and returns the lowest-cost sampled plan. It requires no gradients and works with a forward-model interface, but finite-budget coverage deteriorates rapidly as horizon and action dimension grow.
• CEM iteratively refits a proposal, here a diagonal Gaussian, to elite low-cost samples. This concentrates later evaluations in regions found promising, but a single proposal can collapse early or average incompatible modes. Its advantage over random shooting must be measured under matched budgets.
• MPPI perturbs a nominal action sequence and forms an importance-weighted update under a stochastic-control derivation. CEM uses a hard elite set instead. Their relative performance is problem- and budget-dependent.
• Model predictive control solves a finite-horizon problem from the current state estimate, applies the first action or a short block, and replans. This receding-horizon feedback can correct some observed disturbances and mismatch, but stability and feasibility require additional conditions.
• Replanning closes the loop by incorporating a new state estimate before choosing later actions. In the paired experiment, the same injected noise produced a larger final error open loop than under MPC; the magnitude is specific to the seed and budgets.
• Horizon length trades omitted delayed consequences against search cost and accumulated model error. Maneuver timescales provide an initial guess, while repeated closed-loop sweeps should check performance, constraints, and timing.
• Finite penalties soften constraints and cannot guarantee feasibility. Rejection, feasible parameterizations, constrained optimizers, and independently verified safety layers provide different alternatives under different assumptions. Discrete rollout checks can also miss violations between time steps.
• Warm starting from the shifted previous plan often gives the optimizer a useful proposal at the next tick. Its benefit depends on how quickly the state, goal, constraints, and model change, so it should be compared with a cold start across matched seeds and budgets.
• Real-time discipline is as important as algorithmic choice. The controller must hit its deadline, degrade gracefully when it cannot, and treat plan quality and plan latency as two halves of the same design problem.

The next chapters continue the arc. Differentiable planning uses gradients through sufficiently smooth models to optimize actions, sometimes alongside sampling. Belief-space planning treats uncertainty over latent state as part of the planning problem. The decision-centric lineages of Part VIII then combine learned models with several distinct planning and policy-learning structures.