Games and Learned Game Engines › Long-Horizon Consistency and Rules
World Models HandbookAll chapters Games and Learned Game Engines › Long-Horizon Consistency and RulesOne section · Sections in this chapter5 / 10 ↓
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
One-step accuracy is not enough when a model is fed its own predictions. A small error may change the next input, which can create further error; whether the deviation grows, shrinks, or changes decisions depends on the learned transition and the states visited. The bound below shows one possible relationship between one-step and recursive errors under an explicit smoothness assumption. It is not a universal forecast that every video model fails after a fixed number of frames.
Compounding Error: The Math
Suppose the true dynamics are and a model rollout is under the same action sequence. Let and let be the one-step model error evaluated at the true state. If is -Lipschitz in its state input along both compared trajectories, the triangle inequality gives
where:
- : the error between the predicted and true state at rollout step
- : the Lipschitz constant of with respect to its state input, i.e., the worst-case factor by which a small state perturbation is amplified in one model step
- : the one-step model error at step , i.e., the error introduced when the model predicts the next state from the true current state
Unrolling the recursion gives the error at horizon :
Here is the initial error. If every , then for the bound approaches as grows while the initial-error term decays; for it is ; and for it may grow exponentially. This is an upper bound, not an equality or a measured growth law. Its premises require the same actions and a Lipschitz bound covering the compared states, including the model-generated ones; a bound measured only on training states would not suffice for an out-of-distribution rollout.
The three regimes can be compared directly by iterating the recursion with a constant per-step error.
# Theoretical rollout-error envelopes from e_{t+1} = L * e_t + eps_t,
# with a constant per-step error eps = 0.05 and initial error e_0 = eps.
# L < 1 damps prior error toward a finite bound, L = 1 accumulates
# linearly, and L > 1 permits exponential growth of the bound. These
# curves iterate the bound above; they are not measurements, so no seed is needed.
lipschitz_values = [0.8, 1.0, 1.2]
per_step_error = 0.05
theoretical_horizon = np.arange(0, 21)
theoretical_error_curves = {}
for L_val in lipschitz_values:
errs = np.empty(theoretical_horizon.size)
errs[0] = per_step_error
for t in range(1, theoretical_horizon.size):
errs[t] = L_val * errs[t - 1] + per_step_error
theoretical_error_curves[L_val] = errs
The analytical curves illustrate possible amplification. They do not estimate this fitted grid-world model's Lipschitz constant. Shortening a rollout limits the number of recursive predictions, and re-anchoring replaces a predicted state with an observation; whether either improves decisions still requires evaluation.
Two Failure Modes: Drift and Collapse
Two useful descriptions of long-horizon failure are gradual drift and abrupt rule failure. They are not exhaustive or tied to a universal time scale.
- Drift. A predicted trajectory can gradually diverge in position, appearance, or timing while each frame still looks plausible. Measure it against a reference trajectory under a specified action sequence; do not infer a fixed failure frame count from a single demonstration.
- Rule failure. A model may generate a transition that violates a specified mechanic, such as a character jumping again in mid-air when the game permits only one jump. The error may appear abruptly or after a long rollout; it need not be caused only by an out-of-distribution input.
Different measurements reveal different problems. A frame-distance curve may reveal accumulating prediction error, while a targeted rule test can expose a failure that looks visually minor. Noise augmentation, broader training coverage, longer context, and explicit constraints are possible interventions, but their effects depend on the model and rule being tested.
Diffusion does not make temporal errors disappear: an autoregressive diffusion world model still conditions future frames on generated history. DIAMOND's project report shows a concrete rule violation—repeated jumps not allowed by the reference game. GameNGen used conditioning augmentations for stable autoregressive generation, and its authors evaluated clips after five minutes of generation. Neither example supports a blanket claim that visual generators fail after 30–60 frames or that one model family is intrinsically immune to collapse.
Rule Consistency: The Underappreciated Signal
Games have mechanics that can be stated and tested. A learned model that lets an agent pass through an impassable wall may support a policy that fails in the reference game. But scores can legitimately decrease in some games, and a visible inconsistency does not guarantee that a trained agent will exploit it. To assess rule consistency, specify the rule, action distribution, and observation conditions, then count violations over evaluated transitions.
This idea is worth stating as a criterion.
A learned game engine is rule-consistent on a distribution of trajectories if the fraction of transitions that violate a specified rule of the underlying game is small. Rule consistency is a separate requirement from perceptual similarity: a model can look right while breaking a rule that the eye does not check.
Downstream agent return in the reference game is a separate, indirect test of decision usefulness: poor transfer could stem from rule errors, reward errors, policy optimization, or another mismatch. It cannot by itself identify which mechanic failed. Direct tests can replay a chosen action sequence and score specified mechanics. Part XI's planned chapter on state, physics, causality, and memory evaluation develops that distinction further.
Techniques for Long-Horizon Consistency
Several techniques can be tested for longer rollouts; none is universal.
- Exposure to predicted states. Training on some model-generated histories can reduce the mismatch between training and rollout inputs, if those histories are representative. Do not assume every cited system uses the same procedure.
- Conditioning augmentation. Corrupting conditioning frames during training may improve robustness to prediction noise; GameNGen reports this intervention. It can also change one-step accuracy, so both should be measured.
- Temporal context or external memory. Access to earlier observations can help recover hidden or revisited details, at a model-dependent compute cost. A demonstrated one-minute video is not evidence of a one-minute input context window. See Temporal State and Memory.
- Latent prediction. A compact latent can omit pixel detail irrelevant to a decision objective; it does not make state or reward prediction error vanish.
- Re-anchoring. Replacing a predicted state with a new observation truncates the executed model rollout. It cannot retroactively remove errors in the candidate futures that selected the preceding action.
In the mathematical bound, a shorter horizon reduces the number of propagated terms. The other techniques may change the local errors or dynamics but do not guarantee a smaller or . They need before-and-after evaluation under the same protocol.
Measuring Drift in Our Grid-World Engine
We now collect fifty new random-policy episodes with a separate seed and compare model-generated observations with the reference environment under the same recorded actions, for at most twenty steps. Episodes terminate at different times, so the number contributing to each horizon can change. This is a fresh-episode test, unlike the transition-level validation split above.
def recursive_rollout(model, obs0_flat, actions):
"""Roll the model forward autoregressively. Returns predictions for t=0..T."""
preds = [obs0_flat.detach().clone()]
cur = obs0_flat.unsqueeze(0)
with torch.no_grad():
for a in actions:
a_t = torch.tensor([int(a)])
next_obs, _ = model(cur, a_t)
preds.append(next_obs.squeeze(0))
cur = next_obs
return torch.stack(preds)
def collect_held_out_episodes(n_episodes=50, seed=999):
env = GridWorld(seed=seed)
episodes = []
for _ in range(n_episodes):
obs = env.reset()
ep_obs, ep_act = [obs], []
done = False
while not done:
a = int(env.rng.integers(0, 4))
next_obs, _, done = env.step(a)
ep_act.append(a)
ep_obs.append(next_obs)
obs = next_obs
episodes.append(
(
np.array(ep_obs, dtype=np.float32),
np.array(ep_act, dtype=np.int64),
)
)
return episodes
held_out = collect_held_out_episodes()
T = 20
horizon_err_sum = np.zeros(T + 1)
horizon_counts = np.zeros(T + 1)
with torch.no_grad():
for ep_obs, ep_act in held_out:
obs0 = torch.from_numpy(ep_obs[0].reshape(-1))
preds = recursive_rollout(W_model, obs0, ep_act[:T])
L = len(preds)
true_obs = torch.from_numpy(ep_obs[:L].reshape(L, -1))
errs = ((preds - true_obs) ** 2).sum(dim=-1).sqrt().numpy()
horizon_err_sum[:L] += errs
horizon_counts[:L] += 1
horizon_errors = horizon_err_sum / np.maximum(horizon_counts, 1)Rollout L2 error by horizon: [0. 1.001 1.113 1.182 1.226 1.296 1.334 1.401 1.444 1.469 1.527 1.58 1.607 1.669 1.746 1.815 1.929 2.065 2.201 2.41 2.585]
The mean L2 distance rises from around one after the first predicted transition to about 2.6 at step twenty in this seeded run. At each horizon the mean uses episodes still available there, so the changing set is another reason not to read the curve as a fitted growth law. It neither identifies the model's Lipschitz constant nor proves that a trajectory left the training-state distribution.
The aggregate curve hides the appearance of an individual trajectory. For one selected held-out episode, we decode the highest-scoring agent cell in each predicted frame and compare it with the true path. This illustration does not estimate how frequently such divergences occur.
# Decode the agent cell from channel 0 of each predicted observation and
# compare it with the true agent cell from the held-out episode.
# Selection rule: keep episodes with at least 15 actions; choose the episode
# whose true path visits the most distinct cells (first index breaks ties).
# Episodes come from collect_held_out_episodes(seed=999) collected above.
trajectory_rollout_horizon = 15
candidate_episodes = [
idx for idx, (ep_obs, ep_act) in enumerate(held_out) if len(ep_act) >= 15
]
# Selection uses only the true path, not model error; the figure is illustrative.
candidate_episodes = sorted(
candidate_episodes,
key=lambda i: len(
np.unique(
[
tuple(np.unravel_index(int(c.argmax()), (5, 5)))
for c in held_out[i][0].reshape(-1, 2, 5, 5)[:, 0]
],
axis=0,
)
),
reverse=True,
)
selected_episodes = candidate_episodes[:1]
trajectory_episodes = []
with torch.no_grad():
for idx in selected_episodes:
ep_obs, ep_act = held_out[idx]
obs0 = torch.from_numpy(ep_obs[0].reshape(-1))
preds = recursive_rollout(
W_model, obs0, ep_act[:trajectory_rollout_horizon]
)
n_steps = len(preds)
true_agent = ep_obs[:n_steps].reshape(n_steps, 2, 5, 5)[:, 0]
pred_agent = preds.reshape(n_steps, 2, 5, 5)[:, 0]
true_cells = np.array(
[np.unravel_index(int(p.argmax()), (5, 5)) for p in true_agent]
)
pred_cells = np.array(
[np.unravel_index(int(p.argmax()), (5, 5)) for p in pred_agent]
)
goal_cell = np.unravel_index(int(ep_obs[0, 1].argmax()), (5, 5))
trajectory_episodes.append(
{
"episode": idx,
"goal": goal_cell,
"true": true_cells,
"pred": pred_cells,
}
)
The natural questions are: can an agent still use this model to make decisions, and how careful must it be about the drift? The next section addresses that.