Predictive and Generative Objectives › Historical Motivation

Fitting a dynamical model by reducing prediction error predates modern machine learning. System identification developed statistical methods for choosing parameters from discrepancies between observed and predicted outputs [@ljung1999system]. A common criterion is one-step-ahead prediction error: condition on the available measured history, predict the next output, and penalize the residual. Its consistency and efficiency depend on assumptions about model structure, disturbances, excitation, and the estimator.

Related state-space tools made their modeling assumptions explicit. The Kalman filter recursively updates a Gaussian state estimate for linear dynamics with Gaussian noise, and the linear-quadratic regulator is optimal for its specified linear dynamics and quadratic cost. Learned world models replace those restricted maps with nonlinear, high-dimensional functions, but still face choices about state, uncertainty, and the gap between fitting and use.

Classical system identification also taught us a subtle warning. Under model misspecification, parameters that minimize a one-step prediction criterion need not minimize free-running simulation error, especially when the dynamics are weakly contracting or unstable. The one-step criterion evaluates predictions conditioned on measured history, whereas simulation repeatedly feeds the model's own outputs back into its state. Prediction-error and output-error methods make different assumptions about that conditioning and the disturbance model [@ljung1999system]. This distinction is an ancestor of modern multi-step and overshooting losses.

The reason one-step error can mislead is worth stating plainly. A teacher-forced one-step objective conditions each prediction on measured history, but the fitted transition may later be used to propagate a state for many steps without corrective observations. A parameter that slightly improves the conditional one-step residual while biasing the free-running dynamics can therefore win the one-step comparison and lose the rollout comparison. Output-error and simulation-error criteria instead evaluate trajectories generated by the fitted dynamics, bringing the training criterion closer to free-running use. Their assumptions and statistical properties differ, so neither is a universal fix; the important point is that the deployment mode belongs in the objective design.

Learned world models inherited this story when neural networks replaced linear and low-order parametric models. Different lines of work now make different objective choices: PlaNet and Dreamer combine observation, reward, and latent-dynamics terms; short model rollouts limit the region in which learned dynamics are trusted; and JEPA-style models predict learned target embeddings rather than pixels. These choices respond to recurring practical problems: observation losses can spend capacity on nuisance detail, teacher-forced accuracy may not survive recursive rollout, and a pixel metric need not align with the variables that matter for a decision.

As Part I: The World-Model Idea and Part V emphasized, an objective does more than fit outputs: it shapes the state available to the agent. The rest of this chapter asks when that state is sufficient for the intended predictions and decisions.