Diffusion and Flow World Models

Imagine a tabletop robot trying to flick a puck into one of two goals. Suppose its idealized simulator has a deterministic transition once the full physical state, realized contact impulse, surface parameters, and boundary conditions are fixed. Suppose also that an unobserved left-or-right intention selects the corresponding goal with substantial probability, while residual contact variation is small relative to the distance between the goals. The observer records only a coarse command $a_t$, not that intention, exact contact, or other hidden physical variables. Given the recorded position $s_t$ and command $a_t$, the conditional next-position distribution can then have two goal-centered modes with very little probability between them. Mean-square-error regression targets the conditional mean of these possibilities, which is the prediction at the population optimum. For separated, similarly weighted outcomes, that mean can lie in a low-density region that is a poor representative prediction. Point regression then washes away discrete future structure exactly when the distinction matters for a decision.

Diffusion and flow models offer a flexible modern way to address this problem. Instead of predicting one point, they map a random base sample into a draw from a learned conditional distribution over futures. Ancestral diffusion uses stochastic reverse updates, whereas zero-noise Denoising Diffusion Implicit Models (DDIM; Song, Meng, and Ermon, 2021) with $\eta=0$ and flow ODE integration are deterministic once their initial sample, conditioning, model, and solver settings are fixed. All three obtain diversity from different random base samples. When the learned conditional transport represents both modes, those samples can map to distinct outcomes rather than a single average; ancestral diffusion can inject additional randomness during its reverse updates.

Distributional world models predate diffusion. As discussed in Part III, Ch 3: Latent-Variable Representations, latent-variable models and mixture-density transition models can also represent multimodal futures. Some VAE-based systems use compressed stochastic latents and simple priors, which can reduce modeling or sampling cost; lossy compression may discard task-relevant detail, and total sampling cost still depends on the decoder. Autoregressive token models (see Part V, Ch 3: Autoregressive Token World Models) represent multimodality through sequential token sampling, so decoding is serial and earlier sampling errors can feed later predictions. Diffusion and flow instead update all coordinates of their chosen target through repeated denoiser or vector-field evaluations; conditioning may be injected at every evaluation. They do not require a compact latent bottleneck or token-by-token decoder, although particular variants may use either, and iterative sampling adds latency.

The chapter unfolds in four moves. First, we develop conditional denoising dynamics used by many diffusion world models: the corruption process, the training objective, and learned reverse-time sampling. Second, we distinguish diffusing over one-step states from diffusing over whole trajectories and examine rollout cost and planning by inpainting. Third, flow matching recasts generation as numerical integration of an ordinary differential equation, while rectified flow targets straighter transport paths. Fourth, we separate control mechanisms from the latency constraints that decide whether a sampler fits inside a control loop. A worked example then compares point regression, ancestral diffusion, DDIM, and flow matching on a synthetic bimodal transition.

The object we want to model is a conditional distribution $p(x_0 \mid c)$, where $x_0$ is the target (a next state, a latent, or an entire trajectory) and $c$ is the conditioning signal (the current state and action, plus anything else we choose to observe). In a world-modeling context, $c = (s_t, a_t)$ and $x_0 = s_{t+1}$, but we will keep the notation general. The Gaussian construction below applies directly to suitably scaled continuous Euclidean targets, including scalar states, vector encodings, latents, and flattened horizons. Periodic, discrete, manifold-valued, or hard-constrained targets may require a different representation or corruption process.

A standard Gaussian DDPM defines a forward corruption process that progressively corrupts $x_0$. Let $\beta_t\in(0,1)$ be the scalar noise variance at step $t$, define $\alpha_t=1-\beta_t$ and $\bar\alpha_t=\prod_{s=1}^t\alpha_s$, and let $\mathbf I$ be the identity matrix matching the target dimension. With suitable data scaling and a schedule for which $\bar\alpha_T$ is near zero, the terminal marginal is close to a Gaussian base. A network then learns approximate reverse conditionals rather than exactly inverting each realized corruption path. The forward process is Markovian, meaning each step depends only on the immediately preceding state:

The schedule typically starts with small $\beta_t$ values and grows until $\bar\alpha_T$ is nearly zero. Each step multiplies the preceding state by $\sqrt{1-\beta_t}$ and adds noise with amplitude $\sqrt{\beta_t}$, equivalently covariance $\beta_t\mathbf I$. For $0<\beta_t<1$, the marginal signal-to-noise ratio $\bar\alpha_t/(1-\bar\alpha_t)$ decreases with $t$. Low-noise levels preserve fine detail and are traversed near the end of reverse sampling; sufficiently high cumulative noise makes the terminal marginal close to a simple Gaussian base. A useful bookkeeping identity, derived by composing Gaussians, gives the marginal distribution of $x_t$ directly from $x_0$:

Equivalently, $x_t = \sqrt{\bar\alpha_t}\,x_0 + \sqrt{1 - \bar\alpha_t}\,\epsilon$ with $\epsilon \sim \mathcal{N}(0, \mathbf{I})$. This closed form makes training tractable by jumping directly from $x_0$ to $x_t$. Without it, producing $x_t$ would require simulating $t$ forward transitions, about $T/2$ on average under uniform timestep sampling, which would be prohibitively expensive for the thousand-step schedules used in early DDPM experiments.

The closed-form marginal above lets us visualize the forward corruption process directly. The plot below shows several noisy trajectories along a linear schedule, together with the signal amplitude $\sqrt{\bar\alpha_t}$.

At sampling time we start from a new Gaussian draw and apply learned reverse-time transitions; this approximates the time reversal of the forward distributional dynamics rather than undoing a realized noise path. For sufficiently small $\beta_t$ and a regular noisy marginal density, the local conditional $q(x_{t-1}\mid x_t,c)$ is approximately Gaussian; by contrast, $q(x_{t-1}\mid x_t,x_0)$ is exactly Gaussian for this forward process. Define $\alpha_t=1-\beta_t$. In the common epsilon parameterization, the reverse-model mean is

With the fixed-variance choice used in this chapter's code, sampling iterates $x_{t-1} = \mu_\theta(x_t, t, c) + \sqrt{\beta_t}\,\eta$ with $\eta \sim \mathcal{N}(0, \mathbf{I})$ for $t > 1$, then uses the mean at $t=1$. Other DDPM implementations use the forward-posterior variance $\tilde\beta_t$ or a learned variance. Fresh noise makes ancestral paths stochastic; deterministic samplers instead obtain diversity solely from their initial draw. A useful denoised estimate is

This quantity estimates a clean target compatible with the noisy sample; it does not recover a unique originating example. Samplers can be written using data, score, noise, or velocity predictions. These forms are algebraically convertible when they represent the same exact field under one schedule, but separately trained finite-capacity implementations need not behave identically. Some updates are deterministic, some add noise, and multistage solvers may spend several network evaluations per accepted step. We revisit these choices in the sampling-tradeoff section and implement DDPM and DDIM in the worked example.

To model $p(s_{t+1}\mid s_t,a_t)$ rather than the marginal next-state distribution, the network must receive the relevant conditioning. The simple formulation here uses the current state and action, but a one-step model may also condition on histories, beliefs, observations, or goals. Implementations concatenate these features or inject them through mechanisms such as cross-attention. When a learned score accurately represents a separated bimodal conditional, its reverse-time dynamics can transport different base samples toward different modes instead of returning only their conditional mean.

When the goal is to represent multimodal conditional futures, a distributional predictor or sampler is more suitable than a single MSE point estimate. A design question then appears: diffuse over what? Temporal horizon is one axis, with one-step successor diffusion and joint finite-horizon diffusion as common endpoints and multi-step chunks between them. Separate axes specify which variables are generated, such as states or actions, and whether the target is observable, compressed, or hierarchical and latent.

The first option, which we will call one-step or per-state diffusion, generates one next state or latent at a time. In the Markov version, conditioning is $c=(s_t,a_t)$ and each rollout step draws $s_{t+1}\sim p_\theta(\cdot\mid s_t,a_t)$; histories or belief states can be added when the current state is not sufficient. Relative to generating an $H$-step trajectory jointly, the target omits the multiplicative horizon dimension. That makes it a natural probabilistic transition interface, but not automatically cheap: latency still depends on state dimension, architecture, and sampler.

Per-state diffusion inherits the rollout-shift problem flagged in Part IV, Ch 5: Stochasticity and Uncertainty. Transition uncertainty and model error propagate through later conditioning; they do not simply add as independent variances. If training uses only dataset states while deployment conditions recursively on model samples, errors can move the rollout into contexts where the learned conditional has not been validated. The risk generally grows with horizon, though its rate depends on stability, calibration, and data coverage.

The second option is trajectory diffusion. Here $x_0$ is an entire horizon, typically a matrix $\tau\in\mathbb{R}^{H\times d}$ stacking states and sometimes actions. Diffuser (Janner et al., 2022), for example, uses a temporal U-Net over joint state-action trajectories. Both designs can represent cross-time dependence. Joint trajectory models parameterize it directly and can represent dependencies missed by a one-step model when that model's state or history is not Markov-sufficient. Recursively composed one-step kernels usually induce it through later transitions' dependence on earlier states. Given an initial-state distribution and either a fixed action sequence or a policy, an exact sufficient-state Markov kernel determines the corresponding joint trajectory law. Joint generation also exposes the full horizon to one generative computation, but neither formulation by itself guarantees dynamically feasible or constraint-satisfying plans.

Trajectory diffusion makes horizon-wide inpainting direct because every constrained time index belongs to one generated variable. We can fix $s_0=\bar{s}_0$, reimpose selected states or actions during each denoising update, and generate the remaining entries. A standard causal one-step model needs an additional planner, smoother, bridge-conditioning scheme, or other coordination mechanism to impose comparable future constraints. Decision Diffuser (Ajay et al., 2023) conditions state-sequence generation on returns, skills, or constraints and uses inverse dynamics to recover actions. Diffusion Policy (Chi et al., 2023) is instead an observation-conditioned diffusion policy over action sequences. These are adjacent uses of diffusion, not interchangeable definitions of a predictive world model.

The tradeoffs between the two designs are real:

• Per-state generation omits the horizon factor in its output and can accept either Markov or history-based context, but it must be invoked recursively for a rollout.
• Trajectory generation represents a joint distribution over the horizon and supports direct inpainting, but its output dimension, memory, and per-evaluation cost grow with horizon length.
• Per-state composes naturally with model-based RL pipelines that alternate between sampling transitions and planning; trajectory models equipped with return, goal, or constraint control are closer to generative planners and can replace a separate classical trajectory optimizer by turning planning into guided or constrained sampling, as in Diffuser.
• Both designs depend on training support. A trajectory constraint far outside the dataset can produce an implausible completion even when its masked entries are enforced exactly.

A useful mental model is that per-state diffusion parameterizes a stochastic transition kernel, whereas trajectory diffusion acts more like a generative trajectory optimizer when sampling is guided or constrained by a planning objective. Without such control, it remains a joint trajectory generator. Either design can be placed in a receding-horizon loop: propose a short plan, execute its first action, observe the environment, and replan. Combining joint proposals with a separate reactive transition model is possible, but it is a system design choice rather than an intrinsic property of either method.

Actions can appear either inside the generated variable or in the conditioning. Joint state-action trajectory generation returns a plan and executable actions, as in the original Diffuser formulation. A predictive transition model instead conditions on candidate actions supplied by an external policy or planner. A state-only trajectory generator may use inverse dynamics afterward to infer actions, as in Decision Diffuser. Diffusion Policy generates action sequences directly from observation context and is a policy rather than a state predictor. We return to these roles in Part VI, Ch 2: Actions, Inverse Dynamics, and Latent Actions.

Diffusion models have demonstrated strong generative performance, but standard formulations often use many reverse evaluations and expose choices such as schedules and variance parameterizations. Closely related simulation-free velocity-regression approaches appeared in parallel as Flow Matching (Lipman et al., 2023), stochastic interpolants (Albergo and Vanden-Eijnden, 2023), and rectified flow (Liu et al., 2023). Flow matching trains a vector field for an ordinary differential equation. It removes stochasticity from the integration dynamics, not from the generated distribution, and it retains choices about probability paths and numerical solvers.

Flow notation now changes: let $z_0$ denote a Gaussian base sample and $x_1$ a data sample, rather than reusing diffusion's clean-data symbol $x_0$. We seek a time-dependent vector field $v_\theta(x,t,c)$ such that integrating

from $t=0$ to $t=1$ transports the Gaussian base distribution toward $p(x_1\mid c)$. Even an exact field generally needs numerical integration; one Euler step is exact only for sufficiently simple characteristics. Endpoint samples also do not identify a unique pointwise velocity target. Conventional continuous normalizing flows can be trained through likelihood objectives, but differentiation through ODE integration is costly.

Flow matching supplies supervised velocity targets by choosing a probability path. For the simple independent coupling used here, choose a linear interpolation between $z_0\sim\mathcal{N}(0,\mathbf{I})$ and a data pair $(x_1,c)\sim p_{\mathrm{data}}$:

The conditional flow matching loss is then

where $t\sim\mathrm{Uniform}[0,1]$. The target is the constant conditional-path velocity $x_1-z_0$. Under the conditional flow-matching assumptions, the population MSE optimizer is $v^*(x,t,c)=\mathbb E[x_1-z_0\mid x_t=x,t,c]$; this state-conditioned average is the marginal velocity field for the chosen probability path (Lipman et al., 2023, Theorem 2). Other variants use different paths or non-independent couplings.

Sampling is now a deterministic ODE integration. To keep solver iterates distinct from the endpoint symbols above, let $y_0=z_0$ and set $\Delta t=1/N$ for $N$ uniform forward Euler steps:

from $i=0$ to $N-1$. Given $z_0$, the numerical trajectory is deterministic, and $y_N$ is an approximate target draw with error from field estimation, optimization, and integration. Prediction diversity comes from sampling different $z_0$ values, so the model is a deterministic conditional transport map driven by a random input.

Flow matching is attractive when its learned characteristics can be integrated accurately at low cost. For single-evaluation Euler, NFE equals the number of solver steps; multistage methods such as Heun use more than one field evaluation per accepted step. The optional reflow procedure pairs base samples with endpoints generated by a trained rectified-flow model and retrains on that model-induced coupling; repeated reflow can produce increasingly straight paths and improve coarse-step integration (Liu et al., 2023), but it does not guarantee a monotone reduction in every curvature or intersection measure. There is likewise no general per-round halving rule or portable step count. Required NFE depends on the learned field, coupling, solver, tolerance, model error, and task.

Consistency models (Song et al., 2023) train outputs to agree along a probability-flow trajectory, supporting one-step or few-step generation. Progressive distillation (Salimans and Ho, 2022) repeatedly trains a student to replace two deterministic diffusion-sampler steps with one. These methods reduce evaluation count in their demonstrated settings, but real-time control still requires wall-clock measurements on the target model and hardware.

The formalisms are related but not interchangeable. Under the regularity assumptions of the continuous-time diffusion-SDE formulation, a diffusion SDE admits a probability-flow ODE with the same ideal one-time marginals, and Flow Matching can train vector fields for diffusion probability paths. That equivalence does not make finite-capacity networks, objectives, pathwise trajectories, or discretized samplers identical. In practical flow models, path, coupling, parameterization, and architecture can affect training behavior, learned-trajectory curvature, and discretized sampling quality; these effects must be measured on the target benchmark. Rectification targets straighter characteristics, while distillation performance is separately method- and benchmark-dependent.

For goal-directed planning, a generative model's utility depends strongly on the control information or inference-time objective available to select among its possible futures. An unconditional sampler still has uses in forecasting, simulation, uncertainty estimation, and representation learning; a model conditioned or guided by actions, goals, or constraints can participate more directly in planning. Diffusion and flow world models offer several control mechanisms. The choice shapes behavior and may change sampling cost: direct conditioning can preserve the update schedule, while classifier-free guidance, external guidance, and inpainting add mechanism-specific work.

Conditioning is the first mechanism. Putting a state-action pair, observation history, or belief state into $c$ turns a marginal generator into a conditional predictor. In closed-loop execution, context is refreshed from the latest observation or estimate at each replanning update. In open-loop generation, a horizon is generated from its initial context. A supplied action sequence can condition the full horizon either as an explicit input or, when actions are trajectory variables, by fixing or inpainting those action entries; if the action entries remain unconstrained, the model generates them. Direct context conditioning is a modeling and training mechanism: it changes the distribution the network learns, while the numerical sampler can remain unchanged.

Classifier-free guidance (CFG, Ho and Salimans, 2022) is the second mechanism. Train the network to handle both conditional and null conditioning by randomly dropping $c$ during training. At sampling time, form an affine combination of the null and conditional predictions with $w\geq0$:

Under this convention, $w=0$ gives the null prediction, $w=1$ gives the ordinary conditional prediction, and $w>1$ extrapolates beyond it. Increasing $w$ can strengthen adherence to the condition, but it can also reduce diversity and bias samples away from the learned conditional distribution. The useful setting is empirical: neither small $w$ nor large $w$ guarantees better physical accuracy. The guidance term scales the conditional-minus-null difference by $w$ and amplifies it beyond the ordinary conditional prediction only when $w>1$.

Classifier guidance (Dhariwal and Nichol, 2021) adds the gradient of a noisy-input classifier log likelihood to the diffusion score or reverse update. Diffuser adapts this guided-sampling pattern to control by using the gradient of a learned trajectory-return model to steer samples toward higher predicted return. The guidance model can be changed at inference time, but it introduces another source of error. An inaccurate gradient, applied repeatedly across denoising steps, can bias samples toward trajectories the guide scores highly even when the dynamics model has little supporting data there.

For trajectory diffusion, the third mechanism is direct horizon-wide inpainting. Mask the generated trajectory, reimpose known entries such as the initial state or goal after each update, and sample the rest. A standard causal one-step model instead needs an additional coordination mechanism for comparable future-time constraints. Inpainting enforces the masked values numerically, but it does not guarantee that the interpolated trajectory is dynamically feasible. Constraints outside the learned support can therefore yield a completion with correct endpoints and implausible intermediate states; feasibility must be checked separately.

Latency is the fourth consideration, and it often decides whether a generative world model is usable in a control loop. Neural-network function evaluations (NFE) are a useful cost proxy, but wall-clock time also depends on model size, batching, memory traffic, solver overhead, and hardware. The following ranges are illustrative configurations rather than guarantees:

• An ancestral DDPM may use hundreds or roughly a thousand single-stage updates.
• A deterministic DDIM schedule may use tens of selected timesteps.
• Higher-order diffusion or ODE solvers can reach useful accuracy in tens of steps on some tasks, although a multistage step can require multiple NFE.
• Rectified flows may tolerate coarse integration when their learned paths are sufficiently straight.
• Consistency models and distilled students explicitly target one-step or few-step generation.

The controlling factor inside a model-predictive control loop is wall-clock budget, not NFE alone. A controller running at 10 Hz has a 100 ms cycle for sensing, inference, planning, and action selection together. Reducing evaluations and using a smaller network can both help, but their effect must be measured on the target hardware.

There is therefore an empirical quality-latency frontier, not a universal ordering of samplers. A solver that is better at equal step count may use more network evaluations per step, while a distilled student can trade fidelity for low latency. A practical workflow is to define the wall-clock budget, benchmark candidate samplers at that budget, and then decide whether distillation is worth its training cost. We return to the planning side in Part VII, Ch 1: Sampling-Based Planning and Model Predictive Control.

The final tradeoff is about what is generated. Per-state prediction has a smaller output but repeatedly conditions on model-generated states. Trajectory generation models cross-time dependencies jointly, while multiplying output dimension by the horizon. Short-window generation with frequent replanning is one useful compromise; the best window length is task- and latency-dependent.

To make this concrete, we will build a minimal one-dimensional conditional testbed in which the MSE-optimal point estimate lies between two modes. The state $s_t$ and action $a_t$ are scalars. Given $(s_t, a_t)$, the next state is

with $m$ and $\epsilon$ independent, $\Delta = 1.2$, and $\sigma = 0.25$. The hidden variable $m$ is never observed. Thus $p(s_{t+1}\mid s_t,a_t)=\tfrac12\mathcal N(s_t+a_t-\Delta,\sigma^2)+\tfrac12\mathcal N(s_t+a_t+\Delta,\sigma^2)$. A single point cannot represent both components. Under squared error, its population optimum is the conditional mean $s_t+a_t$, halfway between them in a region of extremely low, but nonzero, density. This mirrors the puck example from the introduction: the dynamics are simple conditional on the hidden mode, and the difficulty is that the mode is unknown to the model.

This testbed also separates marginal and conditional diagnostics. In the data generator below, $s_t\sim\mathcal N(0,1)$ and $a_t\sim\mathcal N(0,0.5^2)$ independently of $m$ and $\epsilon$. After integrating over $s_t+a_t$, the analytic marginal is only weakly bimodal: its center is about $0.63\%$ below its nearby maxima. A coarse marginal histogram can therefore look like one broad peak even though every fixed-context conditional has two well-separated components. Evaluating only the marginal can hide the conditional ambiguity that matters for prediction.

We begin with imports and dataset generation.

Here the measured flow errors at 4, 8, 16, 32, and 64 evaluations are approximately $0.209$, $0.185$, $0.159$, $0.156$, and $0.158$; the DDIM errors are $1.139$, $0.313$, $0.150$, $0.104$, and $0.087$. Euler flow is markedly better at the two coarsest budgets, the curves cross around 16 evaluations, and DDIM is better at the larger budgets. This single seeded, one-dimensional experiment supports a local quality-versus-NFE comparison only; it does not establish a general ordering between flow and diffusion samplers. It is not a latency measurement: deployment requires end-to-end timing on the target control stack.

This scalar demonstration does not expose several system-level issues that matter in deployment. Three important examples are wall-clock latency, decision-making from predictive distributions, and distribution shift during learned-model rollouts.

The first complication is sampling cost interacting with control frequency. Twenty evaluations of a small MLP and twenty evaluations of a large transformer over image-conditioned latents have very different wall-clock costs. At control rates of 10 to 100 Hz, the full perception-planning-action loop has only 100 to 10 ms per cycle. Practical responses include reducing NFE, distilling to a few-step student, shrinking the network, or predicting a short buffer of future states. The relevant comparison is therefore sample quality at a measured end-to-end latency on the target hardware, not the quality of an unconstrained offline sample.

The second complication is that stochastic predictions require distribution-aware evaluation and planning. Planning with deterministic dynamics uses one modeled successor for each state-action pair, whereas planning in an MDP reasons over transition distributions and planning in a POMDP additionally reasons over belief distributions. Sampling-based methods can approximate planning computations in any of these settings. When a diffusion model supplies samples to a separate planner, that planner generally needs an objective-aware way to aggregate downstream outcomes, such as value estimation or a tractable surrogate. Conditional generative planners may instead encode part of that decision rule through return, goal, skill, or constraint conditioning. We revisit the compute and calibration demands of these interfaces in Part VIII: Decision-Centric Research Lineages.

The third complication is distribution shift. During recursive rollout, model samples become later contexts, and prediction errors can move those contexts beyond well-covered training regions. Behavior there is not validated by in-distribution loss and can become inaccurate or poorly calibrated. Guidance changes the sampled distribution as well: extrapolative CFG weights can trade diversity for condition adherence and may worsen errors under shift, but the direction and size of that effect are empirical. Rollout calibration should therefore be tested jointly with the intended guidance setting.

No single commonly used metric is sufficient for all the uses discussed here. Wasserstein distance can have poor empirical sample complexity in high dimensions, with rates governed by the distribution's effective metric dimension (Weed and Bach, 2019). For kernels whose bound remains uniform with dimension, maximum mean discrepancy has dimension-independent square-root concentration rates (Gretton et al., 2012); in growing dimension, its test power depends on the alternative and bandwidth and can deteriorate in some regimes (Huang et al., 2023). Neither metric necessarily reflects planning relevance. Task-level transfer and closed-loop control metrics are more direct for decision making and are often more expensive and environment-specific because they require execution in a task environment, although simulator-based or offline proxies can be cheaper. Evaluation therefore needs distributional, calibration, latency, and downstream measures selected for the application, connected to Part XI: Evaluation and Understanding. Latency, shift, calibration, and planning interact, so the deployed system must be evaluated as a whole.

Diffusion and flow models add a flexible alternative to earlier probabilistic dynamics models, latent-variable models, and autoregressive sequence models. They can model continuous, multimodal conditionals without a prescribed finite mixture and can express some trajectory constraints through inpainting. Flow matching, better solvers, consistency training, and distillation can reduce evaluation counts in suitable settings, making low-latency deployment more plausible when verified by measurement. Their contribution is not the invention of distributional world modeling, but a useful family of steerable generative parameterizations.

Diffusion and flow world models can replace a single deterministic point prediction with samples from a learned conditional distribution. Conditional denoising dynamics learn approximate reverse transitions through a score-matching objective, so training need not evaluate a normalized density. Conditioning on $(s_t, a_t)$ turns this into a transition distribution $p(s_{t+1} \mid s_t, a_t)$.

Two design choices determine how the model is used. Per-state diffusion is a probabilistic analogue of a transition kernel: its output omits the horizon dimension and composes with a planner, but recursive rollout can introduce context-distribution shift when generated states leave the regions covered during training. Trajectory diffusion samples a whole horizon at once, supports direct inpainting and joint cross-time dependencies, and pays for a larger sampling dimension; neither design guarantees dynamic feasibility.

Flow matching learns the velocity field of a chosen probability path and samples through numerical ODE integration. The integration is deterministic given the random base sample. Rectification can reduce characteristic curvature, while consistency training and distillation target fewer evaluations. The evaluations required for a quality target depend on the model, path, solver, task, and metric; the number that fits within a latency budget additionally depends on implementation and hardware.

Controllability can be introduced through direct conditioning, classifier-free guidance, classifier guidance, analogous reward or value-gradient guidance, and trajectory inpainting. Latency can be reduced through fewer evaluations, better solvers, and distillation. A model's position on the quality-latency frontier is a major factor in its suitability for offline planning, real-time control, or analysis, alongside calibration, feasibility, safety, and system-integration requirements.

The worked example makes the representational distinction visible: an MSE point estimate lies between two conditional modes, while the trained DDPM and flow models place samples near both. Its cost curve is method-dependent rather than monotone dominance: Euler flow is better at 4 and 8 evaluations, and DDIM is better from roughly 16 evaluations in this seeded run. For recursive deployment, evaluate expected distribution shifts; inside a closed-loop controller, also measure end-to-end latency on the target hardware.

The next chapter, Part V, Ch 5: Predictive Embeddings, JEPA, and Energy-Based Models, turns to predictive embeddings and energy-based objectives. Predictive-embedding methods move prediction into learned representation spaces. Energy-based methods instead learn scalar compatibility over candidate configurations and may operate in observation space or in a learned representation. Embeddings alone do not define a multimodal predictive distribution; representing multiple alternatives requires an additional distributional or candidate-generation mechanism, such as stochastic latents, explicit mixtures, multiple heads, autoregressive or diffusion sampling, or energy-based inference over multiple compatible configurations.

Ready to test your understanding? Take this quick quiz to reinforce what you've learned about diffusion and flow world models.