Mathematical and Dynamical Foundations › Implementation: Learning a Transition Model and Stress-Testing Its Dynamics
World Models HandbookAll chapters Mathematical and Dynamical Foundations › Implementation: Learning a Transition Model and Stress-Testing Its DynamicsOne section · Sections in this chapter7 / 11 ↓
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
Let's play the role of a practitioner who has logged thermostat data and wants a world model. We will fit two one-step regression models to the same dataset: a full model that receives the current temperature and the heater command applied during that transition, and a baseline that receives temperature only. We will then evaluate both the way practitioners usually do, with one-step error, and the way that actually matters for a world model, by rolling each model out in closed loop and comparing against the true system's attractor. The experimental design is intentionally austere: linear models, least squares, no tuning. If the dynamical failure appears even here, it is not an artifact of deep learning.
First, build the training data and fit both models by least squares. We fit on the first 1,500 steps of the log and reserve the remaining 500 as held-out data, so the closed-loop stress test below is scored against temperatures the models never saw.
n_train = 1500
T_prev = temps[: n_train - 1]
T_next = temps[1:n_train]
M_applied = modes[1:n_train]
ones = np.ones_like(T_prev)
X_full = np.column_stack([T_prev, M_applied, ones])
X_base = np.column_stack([T_prev, ones])
theta_full, *_ = np.linalg.lstsq(X_full, T_next, rcond=None)
theta_base, *_ = np.linalg.lstsq(X_base, T_next, rcond=None)
resid_full = T_next - X_full @ theta_full
resid_base = T_next - X_base @ theta_base
rmse_full = float(np.sqrt(np.mean(resid_full**2)))
rmse_base = float(np.sqrt(np.mean(resid_base**2)))Full model T' = a*T + b*m + c: a = +0.978 b = +0.602 c = +0.195 one-step RMSE = 0.050 (noise floor is 0.050) Baseline T' = a*T + c: a = +0.721 c = +5.595 one-step RMSE = 0.281
The action-aligned full model estimates the temperature and heater coefficients close to their true values, and , and its one-step RMSE on the training data sits near the noise floor. Both printed errors are in-sample residuals over the 1,499 fitted transitions — we deliberately spend the held-out window on the closed-loop stress test below rather than on a second one-step score. The intercept is less precisely estimated because temperature, heater command, and the constant column are strongly correlated over a narrow limit cycle. The baseline's one-step error is more than three times larger in this run, which is the price of ignoring the hidden mode. Note the baseline's fitted slope: because the heater tends to be on when the temperature is low and off when it is high, temperature and mode are correlated in the data, and the least-squares fit absorbs part of the heater's effect into a distorted temperature coefficient. This is a miniature version of confounding, and it will matter in a moment. The baseline is not merely missing a feature; it has actively misattributed the missing feature's effect onto the feature it does have, so its errors are structured, not random.
Now the closed-loop test. We let each fitted model drive the same hysteresis controller from the same initial condition for 300 steps, feeding each prediction back in as the next input, and compare against the true system over the same window.
def rollout_model(theta, use_mode, horizon=300, temp0=20.0):
"""Closed-loop rollout: model predictions feed back as inputs."""
out = [temp0]
cur, mode = temp0, 0
for _ in range(horizon):
cmd = controller(cur, mode)
feats = np.array([cur, cmd, 1.0]) if use_mode else np.array([cur, 1.0])
cur = float(feats @ theta)
mode = cmd
out.append(cur)
return np.array(out)
eval_start = 1650
true_window = temps[eval_start : eval_start + 301]
roll_full = rollout_model(
theta_full, use_mode=True, temp0=float(true_window[0])
)
roll_base = rollout_model(
theta_base, use_mode=False, temp0=float(true_window[0])
)
final_gap_full = abs(roll_full[-1] - true_window[-1])
final_gap_base = abs(roll_base[-1] - true_window[-1])Closed-loop gap after 300 steps, full model: 0.833 degrees Closed-loop gap after 300 steps, baseline: 0.498 degrees True system temperature std over window: 0.410 Full-model rollout temperature std: 0.426 Baseline rollout temperature std: 0.026
Read the figure alongside the printed numbers, because together they make the chapter's central practical argument. The full model, fitted with the command aligned to the transition it caused, reproduces the true system's limit-cycle regime. Its deterministic trajectory need not remain pointwise synchronized with a noisy held-out trajectory, so the terminal gap is not by itself a fidelity certificate. The baseline does noticeably worse on one-step error, but nothing about that error signals the scale of the failure to come: in closed loop, its distorted coefficients translate into a rollout that never oscillates at all. The fitted temperature coefficient, about 0.72, makes the learned map a contraction, so the rollout converges within roughly forty steps to a spurious fixed point near 20.03 degrees, comfortably inside the controller deadband, where the hysteresis rule keeps the heater permanently off. The model has averaged over the two heater regimes instead of representing them, and the result is systematic dynamical distortion: a constant erroneous temperature rather than the true limit cycle. A planner consuming this model would misjudge both how hot the room gets and how quickly it cycles, and it would do so with total confidence, because the model reports no uncertainty about its own distortion.
This is why the qualification test from Chapter 3 insists on separating open-loop prediction quality from closed-loop decision usefulness. A model can pass every one-step metric and still be useless, or actively harmful, as the substrate for planning. Whenever you train a world model, reserve part of your evaluation for autonomous rollouts compared against held-out true trajectories, and pay attention to whether the model reproduces the character of the system's attractors, not just its short-range accuracy. Concretely: compare oscillation periods, duty cycles, variance levels, and occupancy of state space, the statistical fingerprints introduced in the chaos discussion above. The thermostat needed only one hidden bit to break a naive pipeline; production environments hide far more, and far less legibly.