Sampling-Based Planning and Model Predictive Control › Receding Horizons and Replanning

The horizon length $H$ and the replanning policy (how often the controller solves a fresh problem) are the two knobs that decide MPC's behavior in practice. Neither is free: both trade off computational cost against the quality of the closed-loop behavior, and the right settings depend on the system, the model quality, and the time budget. It helps to think of them as two different ways of spending the same resource. A longer horizon spends compute to see further ahead; a shorter replanning interval spends compute to correct more often. The two are not interchangeable, because foresight and feedback do different jobs, but they compete for the same clock.

The horizon length defines what "long" means in the phrase "long-horizon planning." A short horizon omits consequences beyond its boundary and may fail to value a maneuver needed for later success. In this double-integrator task, $H=2$ exposes only 0.2 seconds of nominal motion. Under the stated cost and finite sampling budget, that limited lookahead produces a more myopic closed-loop result; the subsequent sweep reports what happens rather than assuming a particular overshoot pattern. A longer horizon can expose acceleration and deceleration together, but raises search dimension and gives model error more time to accumulate.

There is a second subtlety. A longer horizon raises the search dimension and gives model error more opportunities to accumulate. Much of its tail will also be replanned before execution, although it still influences the selected first action through the objective. The useful horizon must expose delayed consequences and allow the relevant maneuver to appear in the rollout. Our undamped double integrator has no finite settling time, so $1/\Delta t$ is only ten discretization steps, not a physical time constant. We therefore treat 15 to 25 steps as an empirical range for this particular target, action bound, cost, and controller budget rather than derive it from "settling." For stable physical modes, time constants can guide an initial horizon; in every case, closed-loop sweeps should test sensitivity.

Let us verify this empirically. We sweep the horizon from 2 to 40 and record the final distance to the goal after 40 control steps.

This sweep uses the same noise sequence and initializes each planner with the same seed. That removes between-setting variation in the injected disturbances. It does not isolate horizon perfectly: changing the horizon also changes the action-sequence dimension and therefore how the finite CEM budget searches. The five-setting, one-seed result is illustrative; a performance claim would require repeated seeds and uncertainty intervals.

Now the replanning frequency. Instead of replanning every tick, we solve one MPC problem and execute $b$ steps from it before replanning. Larger $b$ reduces compute per unit time but reopens the door to open-loop drift. We implement a block MPC loop.

In this fixed-seed experiment, the final error varies non-monotonically with block size. The comparison keeps the injected noise sequence fixed, but different block sizes make different numbers of stochastic optimizer calls, so it does not isolate feedback frequency from finite-sample planning variation. Treat the curve as an illustration of a feedback-compute tuning experiment, not evidence that error must rise with block size. A deployment study should repeat planner seeds and choose a block whose closed-loop error and constraint metrics remain acceptable under the available compute budget.

The figure shows two tuning sweeps from individual seeded experiments. Error falls as the horizon lengthens and then flattens in the first sweep, while the block-size outcome is non-monotonic in the second. A plateau or apparent optimum is a hypothesis to test, not a universal stopping rule: verify it across disturbances and seeds, and also check computation time, constraint violations, and task success before fixing either parameter.

A third design question is what to replan from. A measured or estimated point state is enough only when that representation is sufficient for the control problem. Under partial observability, belief-space planning propagates a distribution over latent states or hypotheses, as Part VII, Chapter 3 develops. The objective may then involve expected cost, risk, information gain, or constraints under that belief. Taking an expectation does not generally make the optimization landscape smoother, and minimizing expectation alone can hide low-probability, high-consequence outcomes.

Closely related is the question of what cost to optimize. Goal reaching may use stage and terminal distance terms; trajectory tracking may penalize deviation from a reference; stochastic objectives may average or otherwise aggregate costs over predicted futures. The objective, model, constraints, estimator, and replanning policy are conceptually separable, but they are not independent: changing one can alter feasibility, stability, and the best settings of the others.

Finally, terminal costs connect MPC to value-based planning: a terminal approximation summarizes consequences beyond the explicit horizon. If the terminal term equals the true optimal cost-to-go, the model is correct, and the same admissible actions and costs are used, a one-step Bellman minimization recovers an optimal action. Learned terminal values can therefore shorten the horizon needed in practice, but approximation error, model error, and constraints prevent an exact equivalence. Methods such as TD-MPC combine latent rollout optimization with learned value estimates, while MuZero uses a learned model inside tree search; later chapters distinguish those planning structures.