Diffusion and Flow World Models › Summary

Diffusion and flow world models can replace a single deterministic point prediction with samples from a learned conditional distribution. Conditional denoising dynamics learn approximate reverse transitions through a score-matching objective, so training need not evaluate a normalized density. Conditioning on $(s_t, a_t)$ turns this into a transition distribution $p(s_{t+1} \mid s_t, a_t)$.

Two design choices determine how the model is used. Per-state diffusion is a probabilistic analogue of a transition kernel: its output omits the horizon dimension and composes with a planner, but recursive rollout can introduce context-distribution shift when generated states leave the regions covered during training. Trajectory diffusion samples a whole horizon at once, supports direct inpainting and joint cross-time dependencies, and pays for a larger sampling dimension; neither design guarantees dynamic feasibility.

Flow matching learns the velocity field of a chosen probability path and samples through numerical ODE integration. The integration is deterministic given the random base sample. Rectification can reduce characteristic curvature, while consistency training and distillation target fewer evaluations. The evaluations required for a quality target depend on the model, path, solver, task, and metric; the number that fits within a latency budget additionally depends on implementation and hardware.

Controllability can be introduced through direct conditioning, classifier-free guidance, classifier guidance, analogous reward or value-gradient guidance, and trajectory inpainting. Latency can be reduced through fewer evaluations, better solvers, and distillation. A model's position on the quality-latency frontier is a major factor in its suitability for offline planning, real-time control, or analysis, alongside calibration, feasibility, safety, and system-integration requirements.

The worked example makes the representational distinction visible: an MSE point estimate lies between two conditional modes, while the trained DDPM and flow models place samples near both. Its cost curve is method-dependent rather than monotone dominance: Euler flow is better at 4 and 8 evaluations, and DDIM is better from roughly 16 evaluations in this seeded run. For recursive deployment, evaluate expected distribution shifts; inside a closed-loop controller, also measure end-to-end latency on the target hardware.

The next chapter, Part V, Ch 5: Predictive Embeddings, JEPA, and Energy-Based Models, turns to predictive embeddings and energy-based objectives. Predictive-embedding methods move prediction into learned representation spaces. Energy-based methods instead learn scalar compatibility over candidate configurations and may operate in observation space or in a learned representation. Embeddings alone do not define a multimodal predictive distribution; representing multiple alternatives requires an additional distributional or candidate-generation mechanism, such as stochastic latents, explicit mixtures, multiple heads, autoregressive or diffusion sampling, or energy-based inference over multiple compatible configurations.