Stochasticity and Uncertainty › Multimodal Futures

A failure mode of a naive stochastic model is getting the shape wrong. A single Gaussian cannot reproduce a conditional density with separated modes, even if it matches the first two moments. More data cannot remove that family mismatch. When branch identity affects a decision, a planner needs those modes rather than only their mean and variance.

Multimodality can arise near contact regimes whose branch selection is unresolved by the model: an object may tip left or right, a key may turn or jam, or a foot may slip or grip. It can also arise when another agent chooses among discrete actions or when several hidden states remain compatible with the same observation history. These examples are multimodal only when multiple branches retain appreciable conditional probability; fully specified contact or policy dynamics can remain deterministic or unimodal.

A classic tool is Bishop's mixture density network (1994). For a scalar next state, instead of outputting a single mean and variance, the network emits the parameters of a Gaussian mixture:

$$p(s_{t+1} \mid s_t, a_t) = \sum_{k=1}^{K} \pi_k(s_t, a_t)\, \mathcal{N}\!\big(s_{t+1};\, \mu_k(s_t, a_t),\, \sigma_k^2(s_t, a_t)\big),$$

where:

• $K$: the number of mixture components; a component can cover part or all of a mode
• $\pi_k(s_t, a_t)$: the mixing weight of component $k$, produced by a softmax over the components so the weights are positive and sum to one
• $\mathcal{N}(s_{t+1};\, \mu_k, \sigma_k^2)$: a scalar Gaussian over the next state with mean $\mu_k(s_t, a_t)$ and variance $\sigma_k^2(s_t, a_t)$; a vector state instead requires a covariance parameterization such as an isotropic, diagonal, or full covariance

Together, the components can represent several modes, and the mixing weights describe component mass. Components are exchangeable and need not correspond one-to-one with physical branches: they may overlap, split a mode, or collapse onto the same mode. Maximum likelihood permits mode coverage but does not guarantee it, so initialization, capacity, optimization, and validation all matter.

Other architectures offer different flexible parameterizations of multimodal distributions, while still imposing their own finite-model constraints:

• Categorical latent variables: sample a discrete latent $z \sim p(z \mid s_t, a_t)$, then condition the decoder on $z$. The decoder may itself be deterministic or stochastic. The latent can encode branch choice, but its categories are not guaranteed to match physical modes.
• Autoregressive token models: factor the joint distribution over future observations into a product of conditionals over discrete tokens, and let the chain structure express complex multimodality compositionally. We will develop this in the chapter on autoregressive token world models. For a length-$T$ chain over a fixed vocabulary of $V>1$ token values, the Cartesian product contains $V^T$ possible token sequences, although those sequences need not decode to $V^T$ physically distinct futures or receive useful probability mass.
• Diffusion and flow models: transform a simple noise distribution into samples through learned dynamics. Diffusion models define iterative generative processes and are commonly trained with variational-bound or denoising objectives; normalizing flows provide explicit change-of-variables densities when their transforms are invertible. Both can represent complex high-dimensional multimodality, subject to architecture, training, and sampling limits.

For separated multimodal futures, preserve the modes or sample individual branches rather than substituting a low-density mean. The worked example below shows this failure for a deliberately bimodal construction.