The World-Model Evaluation Ladder › One-step prediction

The one-step test scores a prediction at an observed input without feeding the model's previous prediction back into it. That makes it convenient for regression-style models, but the resulting score does not test recursive simulation or action selection. Teacher forcing also does not make transitions statistically independent; neighboring observations can remain correlated.

Let $T(ds'\mid s,a)$ be a controlled Markov transition kernel. It returns a probability law over successors, not a successor state vector. Let $D$ be a common evaluation distribution over $(s,a)$, and let $m(s,a)=\mathbb{E}_T[s'\mid s,a]$ be the conditional mean. For a deterministic point predictor $f$ with finite squared risk, assume the successor and predictor have finite second moments under this joint law. Then

The first term measures conditional-mean prediction error. The second averages conditional squared noise over the same evaluation law. It depends on $T$ and $D$, but not on the deterministic point predictor. To obtain the identity, write $f-s'=(f-m)+(m-s')$ and expand the squared norm. The conditional expectation of the cross term is zero because $\mathbb{E}_T[s'-m\mid s,a]=0$. Noise is already part of $L_1$; it is not added again to an expected squared error against sampled successors.

Where:

• $s$ and $a$: the joint state-action input drawn from $D$;
• $s'$: the sampled successor drawn from $T(\cdot\mid s,a)$;
• $m(s,a)$: the conditional mean successor;
• $f(s,a)$: a deterministic point prediction;
• $\sigma^2_{T,D}$: the average conditional squared noise norm.

Training data $\mathcal D=\{(s_i,a_i,s'_i)\}_{i=1}^N$ may be collected by a behavior policy and a state-estimation pipeline. The evaluation law can be chosen to match that collection regime or to represent a different use. The conditional-mean decomposition applies to the selected law; it does not establish that this law covers deployment.

On $M$ held-out transitions the empirical score is

Here $j$ indexes the evaluation transitions, $M$ is their count, and each $s'_j$ is the observed successor of input $(s_j,a_j)$. With independent evaluation draws from the specified joint law, conditional on a fixed fitted predictor, this sample average is unbiased for its population risk. Disjoint row indices alone do not establish independence from the fitted predictor or independence among evaluation rows. For trajectory data, the sampling and splitting unit needs to be stated.

Three quantities help interpret the score. Let $\mathcal F$ be a fixed hypothesis class, let $f^*_{\mathcal F}$ minimize $L_1$ over that class when such a minimizer exists, and let $\hat f\in\mathcal F$ be the fitted model.

• Approximation error is $L_1(f^*_{\mathcal F})-\sigma^2_{T,D}$, the best conditional-mean error available in the class.
• Fitting excess is $L_1(\hat f)-L_1(f^*_{\mathcal F})$. Finite-data estimation and imperfect optimization can both contribute to it.
• Evaluation error is $\widehat L_1(\hat f)-L_1(\hat f)$, a signed discrepancy due to the evaluation sample.

These definitions give an exact bookkeeping identity when the minimizer exists:

The final bracket can be negative; the first two excess terms cannot. More training data does not remove approximation error in a fixed misspecified class. Under suitable sampling, identifiability, and consistency assumptions it can reduce estimation excess over the best member, even when the class is misspecified.

For example, let $Y=X$ with $\mathbb E[X]=0$ and $\operatorname{Var}(X)=1$, but allow only constant predictors $f_c(x)=c$. The best constant is zero and its approximation risk is one. Fitting the sample mean on $N$ independent draws gives expected excess risk $1/N$. The approximation error remains while estimation excess vanishes. There is no universal rule that greater capacity always increases estimation error; that tradeoff depends on the class, algorithm, regularization, and data.

For a comparison on common evaluation transitions, uncertainty belongs to the paired loss differences. The marginal standard error of one model's MSE is not the standard error of the difference. A numeric gap such as $0.005$ cannot be judged from $M=800$ alone.

Finally, averaging signed prediction errors is not the same as averaging squared errors. Positive and negative mistakes do not cancel in MSE. A conditional mean can be the optimal point predictor yet fail to represent multimodal futures needed by a particular planner. That is a reason to evaluate a predictive distribution or the downstream task separately, not to claim that smoothing is automatically low-error.

A small two-component oscillator makes the distinction concrete without copying the answer into the predictor. Its true state is $z_t=(x_t,v_t)$, and its transition rotates that vector by $\omega=2\pi/20$. The same misspecified model is used in both tests: it rotates by the correct angle but expands the state by a factor of $1.02$. Teacher forcing resets each input to the true state; open-loop rollout feeds the model's previous prediction back into itself.

The teacher-forced vector MSE is $0.02^2=0.0004$, because every true state has norm one. In open loop the norm becomes $1.02^t$, so the error grows even though the rotation angle is correct. A constant-mean predictor does not track this sine wave under teacher forcing: its scalar MSE across the 41 plotted states is $20/41\approx0.487805$, not zero.

Three confounds deserve special attention.

1. Distribution coverage. The evaluation law $D$ need not match the state-action occupancy induced by deployment. If data cover only near-upright robot states, predictions far outside that region are extrapolations unless other data or structural constraints inform them. A state absent from a finite sample is not necessarily outside the support of the population distribution.
2. Stochasticity. Point-prediction MSE and distributional scores answer different questions. A probabilistic model can be scored by its mean for MSE, or by a suitable proper scoring rule for its distribution. Comparing a random sample from one model with a point estimate from another changes the scoring convention. Uncertainty-aware planning needs separate tests. [Perceptual and Generative Evaluation] is planned to examine observation-quality scoring, not certify a planning system.
3. Observation error. A state estimated from an image or point cloud can be inaccurate before the dynamics predictor receives it. Separating perception and transition errors needs component-specific measurements, identifiable model/noise assumptions, or interventions. Aggregate prediction error alone does not distinguish those components. Their units, directions, dependence, and propagation matter; two nominal error magnitudes alone do not determine the combined prediction error.

A one-step result is therefore a distribution-specific check. It can be decisive if that is the intended task. It cannot, by itself, establish rollout quality, intervention accuracy, or useful control.