What Is a World Model? › Prediction Versus Simulation

With the anatomy in place, we can draw a distinction that sounds trivial and is anything but: the difference between predicting the future and simulating it. The confusion between these two activities is responsible for a surprising share of miscommunication in the field, partly because the word "prediction" is used loosely for both, and partly because simulation looks like repeated prediction, which suggests it should inherit prediction's guarantees. It does not, and seeing why is essential.

Prediction answers a question about the future with an estimate. Depending on need, the estimate takes different forms: a single most-likely next state (point prediction), a full conditional distribution, an expected value of some quantity, or a probability that a threshold is crossed. Prediction is typically local in time: one step, or a few. Each form of answer serves a different consumer: point predictions feed controllers that want a single target, distributions feed risk assessments, expectations feed value estimates, and tail probabilities feed safety monitors. Choosing the form is part of designing the model, not an afterthought.

Simulation produces concrete, sampled future trajectories by chaining the model through time: sample $s_{t+1}$ from the model given $(s_t, a_t)$, treat the sample as the new state, sample again, and repeat. Run the chain many times and you get an ensemble of possible futures; run it once and you get one coherent "imagined experience." Simulation is how an agent rehearses: it is the machinery behind planning by rollout, behind counterfactual reasoning ("what if I had braked?"), and behind the imagination phase of modern model-based agents like Dreamer and MuZero. The phrase "imagined experience" is chosen carefully: a single sampled rollout has the texture of an experience, a coherent sequence of states and events, which is exactly what some learning algorithms, and arguably some biological processes such as hippocampal replay, consume.

The two activities are related but not equivalent, and conflating them causes real engineering damage. Simulation is prediction applied recursively, and recursion has a cost: errors compound. If the one-step model has small error $\epsilon$, a naive hope is that $T$-step error stays near $T\epsilon$. Reality is harsher. Each predicted state is fed back in as if it were real, so the model drifts off the data distribution it was trained on, and errors interact multiplicatively through nonlinear dynamics. A model with excellent one-step accuracy can produce absurd hundred-step rollouts, a phenomenon you will watch happen numerically in the code section below. The mechanism deserves a name: feeding a prediction back as an input is called closed-loop model rollout, and the resulting mismatch between training inputs (real states) and rollout inputs (predicted states) is a form of distribution shift the model was never trained to survive.

This leads to the deepest practical lesson of the chapter, one that separates people who build world models from people who merely train sequence models.

Two corollaries follow immediately. First, open-loop prediction quality is necessary but not sufficient for closed-loop usefulness. Evaluating a world model only on held-out one-step prediction is like certifying a bridge by testing one brick. Second, optimizers exploit models. A planner searching for high-value action sequences will inevitably probe regions where the model is least constrained by data, precisely because those regions often promise unrealistic rewards. Managing this exploitation, through conservatism, ensembles, or uncertainty penalties, is a central theme of Part VII on planning and agency. The exploitation dynamic is worth internalizing as a pattern: wherever an optimizer meets an imperfect model, the optimizer finds the imperfections, because imperfections usually look like opportunity.

A related vocabulary distinction recurs throughout the literature. Open-loop means committing to an action sequence in advance and executing it without feedback. Closed-loop means re-planning or adjusting as observations arrive. Models degrade differently in the two regimes: open-loop execution suffers pure compounding error, while closed-loop use suffers repeated exposure to the model's own biases, corrected partially by fresh observations. Receding-horizon schemes such as model-predictive control sit between the extremes and are the workhorse of Part II, Chapter 4. The reason receding-horizon control works so well in practice is that it converts one long-horizon trust question into many short-horizon ones: instead of trusting the model for a hundred steps, you trust it for ten, then re-anchor on fresh reality, then trust it for ten again. Shortening the trust interval is often the single cheapest improvement available to a struggling model-based system.

Finally, note what simulation uniquely enables: counterfactuals and rare events. A predictor tells you the expected future; a simulator lets you ask what happens under interventions, under adversarial action sequences, or in the tail of the distribution. This is why safety-critical applications lean heavily on simulation, and why the fidelity of the underlying model, not just its average accuracy, becomes the binding constraint. Average accuracy is dominated by common situations, but accidents live in tails, and a model that is 99 percent accurate everywhere except the 1 percent of states where crashes occur is a dangerous model for a safety case, however impressive its aggregate metrics.