The World-Model Evaluation Ladder › Transfer

Now the rung-one evaluation, on 800 i.i.d. transitions.

On these independent standard-normal inputs, each model's residual is a zero-mean Gaussian linear combination of state, action, previous action, and independent process noise. Its population MSE is therefore

The resulting risks are $0.0053$ for Aggressive, $0.0229$ for Conservative, and $0.0404$ for Delayed. Aggressive's state coefficient error is smaller than Conservative's; Delayed matches that coefficient exactly. Aggressive has lower total risk than Delayed because Delayed's spurious previous-action term adds variance $0.04$. The sampled MSEs differ from these population risks, but retain the same order for the stated seed. No candidate was fitted to the sample.

For independent zero-mean Gaussian residuals of variance $v$, the variance of their squared value is $2v^2$. Thus the sample MSE has standard error $v\sqrt{2/M}$: at $M=800$, five percent of the population MSE. The code prints that marginal calculation and separately estimates the standard error of each paired loss difference. The latter includes the covariance between model losses scored on the same transitions.

Now for rung two. We fix $s_0 = 0.3$ and apply constant actions $u_t = 0.2$ over 40 steps, then measure trajectory RMSE.

Under a constant positive push $u = 0.2$, the true system's noiseless steady state is $b^\star u / (1 - a^\star) = (0.50)(0.2) / (1 - 0.95) = 0.1 / 0.05 = 2.0$. The Aggressive model has $1.02 > 1$, so its rollout diverges. The Conservative model has steady state $0.1 / (1 - 0.80) = 0.1 / 0.20 = 0.5$, well below the truth. The Delayed model acquires an effective action gain $b_{\text{m}} + c_{\text{m}} = 0.50 + 0.20 = 0.70$, giving steady state $(0.70)(0.2) / (1 - 0.95) = 0.14 / 0.05 = 2.8$, incidentally closer to the true value than the Conservative model's. Delayed wins rung two, despite losing rung one. The gain $0.70$ is not the true gain $0.50$. Delayed wins this finite-horizon comparison because its state decay coefficient matches the truth and its accumulated gain error is smaller than the competing models' trajectory errors for this initial state and input. This is not a claim that the lagged model has correct constant-input dynamics.

For a controlled-intervention trajectory diagnostic, change the constant input to $u_t=0.4(-1)^t$ over the same 40 steps. This is another open-loop trajectory test under a deliberately changed action sequence. It is not the intervention-response difference $\Delta_H$ defined earlier, and it does not establish causal identification. The simulator supplies the true dynamics by construction.

Delayed wins the 40-step alternating-trajectory test, but not because its steady alternating response is most accurate. After the initial step, $u_{t-1}=-u_t$, so its effective alternating gain is $0.50-0.20=0.30$. For $|a_{\mathrm m}|<1$, substituting $s_t=-K(-1)^t$ into the forced recurrence gives the signed coefficient $K=0.4(b_{\mathrm m}-c_{\mathrm m})/(1+a_{\mathrm m})$. The amplitude magnitude is $|K|$. All displayed stable candidates have $b_{\mathrm m}>c_{\mathrm m}$, so their coefficients and magnitudes agree. The true amplitude is $0.1025641$, Conservative's is $0.1111111$, and Delayed's is $0.0615385$. Delayed instead benefits at this short horizon from matching the true transient decay coefficient. At 500 steps Conservative has lower trajectory RMSE than Delayed, as the printed results show. Aggressive is unstable, so its formal forced periodic solution is not an attracting steady response.

We also compute the actual response-difference test. Use the constant input as baseline and the alternating input as intervention, starting both trajectories at $s_0=0.3$ with previous action zero. Compare their final-state differences, not merely the intervened trajectory:

This response metric and the alternating-trajectory RMSE answer different questions, even if they select the same winner here. Equal additive trajectory biases cancel in the difference, but general dynamical errors need not. The metric tests these two known simulator interventions only; it does not certify unseen interventions or an observationally learned causal model.

Rung four is closed-loop planning. We use a one-step MPC that searches 201 equally spaced actions in $[-0.5,0.5]$ and minimizes squared predicted next-state distance to $s^\star=1$. It then executes the chosen action in the true deterministic environment and reobserves the state. This costs 201 candidate one-step predictions per control step, with no action penalty and no lookahead beyond one step. The grid enforces the action constraint by construction; no state-safety constraint is imposed.

An idealized continuous-action controller helps explain the tracking bias. Assume deterministic dynamics, an inactive action bound, no action penalty, and one-step squared next-state tracking toward target $g$. With $b_{\text{m}}\ne0$, its unconstrained choice is

At a fixed point both state and action are constant. The true environment therefore satisfies $(1-a_{\text{env}})s_\infty=b_{\text{env}}u_\infty$, while the model's target equation becomes $g=a_{\text{m}}s_\infty+(b_{\text{m}}+c_{\text{m}})u_\infty$. Eliminating $u_\infty$ yields

where $s_\infty$ is the continuous controller's equilibrium, $a_{\text{env}}$ and $b_{\text{env}}$ are true environment coefficients, and $a_{\text{m}},b_{\text{m}},c_{\text{m}}$ are model coefficients. The denominator must be nonzero. This algebra finds an equilibrium, not a guarantee of stability or feasibility; those must be checked separately.

The continuous same-environment equilibria are approximately $0.9345794$ (Aggressive), $1.1764706$ (Conservative), and $0.9803922$ (Delayed). In transfer they are $0.8196721$, $1$, and $0.8130081$. The lagged-action term remains in the transfer equation; dropping it would give the wrong Delayed equilibrium.

The implemented controller searches 201 discrete actions instead. It need not converge to those continuous equilibria: grid quantization can produce small periodic trajectories. We report the actual mean absolute tracking error over states 31 through 40, not an analytically inferred bias. Delayed has the lowest same-environment tail error, while Conservative has the lowest transfer tail error. Conservative is worst in the same-environment closed-loop comparison, not least accurate on every metric.

From this initialization, Delayed approaches a five-step cycle in the same environment and a two-step cycle in transfer. The transfer cycle has states approximately $0.8152778$ and $0.8097222$, with corresponding executed actions $0.335$ and $0.315$; its tail MAE is $0.1875$. These grid trajectories explain why a finite-run statistic differs from a continuous-action fixed point. Neither statistic is an architectural ranking.

Closing the loop also introduces a decision baseline. An oracle one-step controller using the true coefficients separates dynamics misspecification from action-grid and startup effects in this toy; a zero-action baseline shows the value of control. This is a diagnostic baseline, not a claim that oracle dynamics are available in deployments.

The actual grid-controller trajectories show the residual tracking errors.

Now compare rankings without dividing by a near-zero winner error. The transfer winner's error is almost zero, so dividing other models' errors by it would create enormous ratios and hide differences on the other tests. We show within-test ranks, with 1 best. Ranks discard the size of the gaps; the printed raw metrics retain that information. Do not sum the ranks into a universal score.

The pattern is exactly what the ladder is designed to expose: different rungs can favour different winners.

• One-step MSE favors Aggressive on the stated independent Gaussian inputs.
• Constant-input rollout favors Delayed at the specified 40-step horizon, despite its incorrect gain.
• The response-difference metric favors Delayed for this baseline/intervention pair. The additional alternating-trajectory test is not the same metric.
• Same-environment grid control also favors Delayed; the example does not show a winner reversal between that rollout and that controller.
• Transfer favors Conservative because its coefficients match the changed environment more closely. This is no general guarantee of robust transfer.

These rankings are conditional on the test distribution, action sequence, horizon, initial state, and scoring rule. A scalar chosen for a declared deployment objective can be useful. Report that choice explicitly, and retain the other diagnostics when they describe additional intended uses.