Mathematical and Dynamical Foundations › Random Variables and Conditional Distributions

A world model is fundamentally a device for answering questions of the form "what might happen next, and with what probabilities?" Before we can talk about transitions and trajectories, we need a precise language for that kind of question. That language is probability theory, and its central objects are random variables and the conditional distributions that connect them. Everything else in this chapter, and most of what follows in this handbook, is built by composing these objects in time.

A random variable is not a variable in the programming sense. It is a formal device for attaching numbers to the outcomes of something uncertain, together with a distribution that says which numbers are more or less likely. The distinction matters because programmers routinely conflate the two: a Python variable holds one value at a time, while a random variable holds a whole space of possible values at once, weighted by belief. When we say a room's temperature tomorrow is a random variable, we are not saying the temperature is fuzzy or ill-defined; we are saying that our knowledge of it is incomplete, and the distribution is a complete accounting of that incompleteness.

When we write $s_{t+1} \sim p(\cdot \mid s_t, a_t)$ later in this chapter, the tilde means "drawn from," and the whole expression says: the next state is a random variable whose distribution depends on the current state and action. Nothing more mysterious than that. The dot inside the distribution is a placeholder for the value being drawn, and the part after the bar lists everything the draw is conditioned on. Learning to parse such expressions fluently pays off immediately, because nearly every formula in this handbook is a statement about who is conditioned on whom.

Two habits worth building now. First, always ask what a distribution is over: temperatures, pixel intensities, latent vectors, actions. Distributions over different spaces behave differently, support different operations, and demand different estimators, and many modeling bugs amount to silently mixing up which space a distribution lives in. Second, always ask what a distribution is conditioned on: a predictive distribution conditioned on the wrong information is worse than useless, because it feels principled while answering the wrong question. A weather model conditioned only on the season will produce confident seasonal averages; conditioned on today's pressure field, it produces something you would actually act on. The difference between those two conditionings is, in miniature, the difference between a bad world model and a good one.

Suppose $X$ and $Y$ are two random quantities, for example today's soil moisture and tomorrow's plant growth. Their joint distribution $p(x, y)$ describes them together: it assigns a probability, or probability density, to every combination of values, and thereby encodes not just how each quantity varies on its own but how they co-vary. Summing or integrating one out gives the marginal:

$$p(y) = \int p(x, y)\,dx.$$

The marginal answers questions about $Y$ alone, deliberately ignoring whatever $X$ might have told us. It is the distribution you are stuck with when you have no access to $X$, and its spread is typically wider than what conditioning would leave you.

The conditional distribution reverses the direction of reasoning. It answers: if I have already observed $X = x$, what should I now believe about $Y$?

$$p(y \mid x) = \frac{p(x, y)}{p(x)},$$

defined whenever $p(x) > 0$. Read as a rule, it says: restrict attention to the slice of the joint where $X$ takes the observed value, then renormalize so the slice sums to one. Conditioning is the mathematical core of every world model. An encoder maps an observation to a state; a dynamics model produces $p(s_{t+1} \mid s_t, a_t)$; a reward head produces $p(r \mid s_t, a_t)$. All of these are conditional distributions, and the quality of a world model is largely the quality of its conditionals. When this handbook later speaks of calibration, uncertainty quantification, or model exploitation by planners, it is always asking the same underlying question: are these conditionals honest?

Conditioning changes shapes, not just centers. If $Y = 0.8X + \varepsilon$ with $\varepsilon \sim \mathcal{N}(0, 0.5^2)$, then knowing $X = 1$ does not merely shift your belief about $Y$ toward $0.8$; it also shrinks its spread from the marginal spread of $Y$ down to $0.5$. Both effects follow from the same algebra: once $X$ is known, the only remaining randomness is the noise term, so the conditional distribution is the noise distribution relocated to the regression line. Let's see both effects empirically.

The reason conditional distributions deserve this much attention is that they compose. Any joint distribution over a sequence can be factored into a product of conditionals, a fact called the chain rule of probability:

$$p(z_1, z_2, \dots, z_T) = p(z_1)\,p(z_2 \mid z_1)\,p(z_3 \mid z_1, z_2)\cdots p(z_T \mid z_1, \dots, z_{T-1}).$$

This identity looks innocuous and is anything but. It says that a full distribution over an entire trajectory can be built from nothing but a sequence of local predictive distributions, each conditioned on the past. There is no additional assumption hiding in it: the chain rule is an identity, true for every joint distribution whatsoever. All of the modeling content enters through the choices we make about what each conditional may depend on, which is exactly where assumptions like Markovianity will enter shortly. Every autoregressive language model, every recurrent dynamics model, and every rollout-based planner in this handbook is an application of the chain rule. When a world model rolls out a prediction for a hundred steps, it is multiplying a hundred conditional distributions together, and the failure modes of that multiplication, which we will quantify shortly, are among the central practical problems of the field.

Expectation is the other workhorse. For a distribution $p(x)$, the expected value is $\mathbb{E}[X] = \sum_x x\,p(x)$ or $\int x\,p(x)\,dx$: the probability-weighted average of the random variable, the single number that summarizes where the distribution's mass sits. Two properties carry most of the weight in practice. The law of iterated expectations, $\mathbb{E}[Y] = \mathbb{E}\big[\mathbb{E}[Y \mid X]\big]$, says that averaging conditional predictions reproduces the marginal prediction, which is why a simulator's average behavior can be correct even when no individual rollout is. This is simultaneously reassuring and dangerous: reassuring because aggregate statistics computed from simulations can be trusted even when individual runs look strange, and dangerous because it means average-case validation can hide pathologies that only appear trajectory by trajectory. And linearity of expectation, $\mathbb{E}[X + Y] = \mathbb{E}[X] + \mathbb{E}[Y]$, holds even when $X$ and $Y$ are dependent, which is why error budgets can often be added component-wise even when error sources interact. That linearity holds without independence is what keeps many applied probability calculations tractable.