Planning, Control, and Policy Evaluation › Planning, Control, and Policy Evaluation

Suppose a learned-model planner predicts a return of $-3.2$ but earns $-48$ when its controller runs in the reference environment. These numbers are hypothetical, not outputs from the notebook below. The contrast matters because a predicted rollout and an executed closed loop answer different questions: the first is about the surrogate dynamics, while the second is about the model, planner, policy, and environment acting together.

A gap can arise from model error, but it can also reveal mismatched evaluation protocols or an implementation bug. It is not inevitable: a model that is exact on the visited trajectories, with matching rewards and controller random streams, can predict return exactly. Lower average one-step loss can nevertheless coexist with worse closed-loop return. Prediction loss averages over its evaluation distribution; a planner may query a different distribution and amplify errors that matter to action selection. Good planning does not require this shift, and unsuccessful planning can shift too. Measure both predicted and executed return before attributing their difference.

This chapter develops that comparison and the resource accounting around it. Chapter 62, Perceptual and Generative Evaluation, covers image-quality and distributional diagnostics. Chapter 63, State, Physics, Causality, and Memory Evaluation, covers state readouts, physical constraints, interventions, and memory diagnostics. Chapter 61 introduced the full evaluation ladder, including closed-loop decisions and transfer. Here we expand the decision-facing tests: what return does a composed controller earn on specified tasks, distributions, and budgets?

We examine four questions. Task return and sample efficiency connect achieved performance to real-interaction budgets. Planning horizon and compute tradeoffs compare search depth and breadth. Model exploitation and policy ranking test selection-sensitive prediction errors and the ordering of fixed candidate policies. Zero-shot, few-shot, and cross-domain transfer specify what stays frozen, what adapts, and which deployment conditions change.

These questions share a composed-system framework, but they are not marginals of one fixed trajectory distribution. Changing a planner, policy, or environment can change the distribution itself; functionally equivalent controllers or changes outside visited states need not do so. Ranking compares scores across policies; it need not analyze distribution tails. Transfer can change just one environment parameter, as it does below. The diagnostics can disagree: a biased return estimate can preserve policy order, and a costly planner can still achieve good return. Joint reporting makes those distinctions visible.

A concrete lesson runs through all four faces. The evaluation target is a composed system, not a component. When we say "the new model is better," we must always be able to say "better at what return, on what initial-state distribution, at what compute and interaction budget, and compared against which baselines on which held-out cases." Without that, the numbers are decoration.

Model quality has several separable dimensions. Predictive fidelity compares dynamics and reward predictions with reference outcomes under a stated distribution. Representation quality asks what task-relevant information a state representation makes accessible. Uncertainty quality includes calibration or coverage of specified predictive quantities, sharpness, and useful discrimination between uncertain and well-supported predictions. Decision usefulness asks what a controller achieves in execution. All four can use environment-derived reference data; the decision test additionally executes actions in a closed loop. Strong predictive or uncertainty scores alone do not certify control performance.

This separates two questions: is the model accurate under the chosen predictive metric, and is it useful for this decision problem? Small distribution-averaged errors can still matter greatly if they affect reward-sensitive states or action ordering. Conversely, errors in task-irrelevant quantities may have little effect on a particular controller. Name the metric and task rather than treating accuracy and usefulness as synonyms.