Taxonomies of World Models › Explicit and Implicit Models

The first and most fundamental question about any world model is: can you ask it a question? An explicit model exposes a prediction interface. Given a hypothetical state and action, it returns a predicted outcome, and you can chain such queries into arbitrary rollouts. An implicit model contains the same kind of knowledge, but fused into a larger computation, so that no component in isolation answers "what happens next?"

Most of this handbook concerns explicit models, because explicitness is what enables the classic uses: planning by lookahead, model-based data augmentation, counterfactual reasoning, and analysis. Classical examples include the analytical simulators of aerospace engineering, the identified state-space models of Part II, and learned forward models like the ones we will build later in this chapter. The defining property is composability: because each prediction returns a state in the same format the model consumes, you can recurse. One-step accuracy becomes, at least in principle, multi-step foresight.

This composability is exactly what made classical model-predictive control possible decades before anyone spoke of world models. An MPC controller at each time step asks the model a short sequence of hypothetical questions: "if I apply torque $u_1$ now, then $u_2$ next step, and so on, where does the plant end up?" The answers form a predicted trajectory, the controller scores it against a cost function, applies the first action, and repeats. Every ingredient of that loop, the rollout, the scoring, the replanning, presupposes an explicit interface. The same is true of classical observers such as the Luenberger observer and the Kalman filter: both maintain an explicit internal estimate of the state and update it using an explicit model of how states evolve. Modern learned world models inherit this entire workflow; only the source of the transition function changes.

Explicitness also carries obligations. An explicit model must decide what its outputs are: pixels, latents, objects, or symbols. That decision is the subject of the next section. And an explicit model can be exploited, meaning a controller can steer it into regions where its predictions are confidently wrong. We treat that failure mode as a first-class concern throughout Part VII and Part XII. The exploitation problem has a classical analogue too: aggressive MPC controllers have always been able to drive a nominal model outside the region where its linearization was valid, and the field's answer, robust and tube-based MPC, anticipates the uncertainty-aware techniques we develop later.

Implicit models are more common than the term suggests. Consider three examples.

• Model-free value functions. A critic trained by temporal-difference learning must, in some sense, know the dynamics: its estimates change in exactly the way the environment's transition distribution dictates. But you cannot query it with a hypothetical state-action pair and receive a predicted next state. The dynamics knowledge is real yet inaccessible. You can observe that the critic assigns high value to a state, but you cannot ask it why in terms of future events, nor redirect it to evaluate a hypothetical intervention the environment never produced during training.
• Successor features compress the transition structure of a Markov reward process into a quantity that supports rapid policy evaluation, again without exposing raw next-state predictions. They encode expected future feature occupancy, which is a summary of dynamics filtered through a discount factor, so the compression is principled but lossy in a direction chosen by the discount, not by the user.
• MuZero (Schrittwieser et al., 2020) is the sharpest modern example. Its "model" operates in a latent space and predicts only three things the agent cares about: reward, value, and policy. It never reconstructs observations. Planning works by searching over imagined latent trajectories, so the model is explicit for decisions while remaining implicit about appearance.

That last phrase suggests the useful refinement: implicitness is not binary but scoped. MuZero's model is explicit about decision-relevant quantities and implicit about everything else. Whether that scope is a strength or a weakness depends entirely on your use case. If your sole consumer is a search procedure maximizing reward, MuZero's scope is close to optimal: it spends every unit of capacity on quantities the planner reads. If any other consumer exists, a debugging engineer, a verification tool, a downstream task with a different reward, the missing scope becomes a liability, because there is no way to interrogate the model about anything beyond its three trained heads.

The tradeoff cuts both ways. Implicit models avoid paying for reconstruction of irrelevant detail, which is why MuZero-style models can be easier to train than reconstruction-based ones: there is no loss term tempting the representation to spend capacity modeling visually salient but decision-irrelevant texture. Explicit models, in exchange for that cost, support uses that implicit models cannot: debugging, verification, transfer of the model alone to new tasks, and the counterfactual queries that safety analysis in Part XII requires. Transfer deserves emphasis because it is easy to overlook: an explicit dynamics model trained on one reward can be re-plugged into a new reward immediately, whereas an implicit model whose value head was shaped by the old reward carries assumptions about objectives baked into its weights, and retraining is usually unavoidable.

A practical rule of thumb: be explicit about whatever you will query, and implicit about whatever you will not. If your agent plans by imagining observations, model observations. If it plans by imagining values, model values. Trouble begins when you guess the wrong scope, for example building a beautiful video predictor for an agent that only ever needed two bits of contact information. The inverse error is just as real: training a purely value-scoped model and then discovering mid-project that you needed to audit its imagination, diagnose a failure, or satisfy a regulator who wants to see what the agent believes will happen. Scope decisions are cheap to make early and expensive to reverse late, which is why we treat them as a first-order design choice rather than an implementation detail.