Diffusion and Flow World Models › Flow Matching and Continuous-Time Generation

Diffusion models have demonstrated strong generative performance, but standard formulations often use many reverse evaluations and expose choices such as schedules and variance parameterizations. Closely related simulation-free velocity-regression approaches appeared in parallel as Flow Matching (Lipman et al., 2023), stochastic interpolants (Albergo and Vanden-Eijnden, 2023), and rectified flow (Liu et al., 2023). Flow matching trains a vector field for an ordinary differential equation. It removes stochasticity from the integration dynamics, not from the generated distribution, and it retains choices about probability paths and numerical solvers.

Flow notation now changes: let $z_0$ denote a Gaussian base sample and $x_1$ a data sample, rather than reusing diffusion's clean-data symbol $x_0$. We seek a time-dependent vector field $v_\theta(x,t,c)$ such that integrating

$$\frac{dx}{dt} = v_\theta(x, t, c), \qquad x(0)=z_0,\quad z_0 \sim \mathcal{N}(0, \mathbf{I}),$$

from $t=0$ to $t=1$ transports the Gaussian base distribution toward $p(x_1\mid c)$. Even an exact field generally needs numerical integration; one Euler step is exact only for sufficiently simple characteristics. Endpoint samples also do not identify a unique pointwise velocity target. Conventional continuous normalizing flows can be trained through likelihood objectives, but differentiation through ODE integration is costly.

Flow matching supplies supervised velocity targets by choosing a probability path. For the simple independent coupling used here, choose a linear interpolation between $z_0\sim\mathcal{N}(0,\mathbf{I})$ and a data pair $(x_1,c)\sim p_{\mathrm{data}}$:

$$x_t = (1 - t)\,z_0 + t\,x_1, \qquad u_t = \frac{d x_t}{d t} = x_1 - z_0.$$

The conditional flow matching loss is then

$$\mathcal{L}_{\mathrm{FM}}(\theta) = \mathbb{E}_{t,z_0,(x_1,c)\sim p_{\mathrm{data}}}\!\left[\left\|v_\theta(x_t, t, c) - (x_1 - z_0)\right\|_2^2\right],$$

where $t\sim\mathrm{Uniform}[0,1]$. The target is the constant conditional-path velocity $x_1-z_0$. Under the conditional flow-matching assumptions, the population MSE optimizer is $v^*(x,t,c)=\mathbb E[x_1-z_0\mid x_t=x,t,c]$; this state-conditioned average is the marginal velocity field for the chosen probability path (Lipman et al., 2023, Theorem 2). Other variants use different paths or non-independent couplings.

Sampling is now a deterministic ODE integration. To keep solver iterates distinct from the endpoint symbols above, let $y_0=z_0$ and set $\Delta t=1/N$ for $N$ uniform forward Euler steps:

$$y_{i+1} = y_i + \Delta t \cdot v_\theta(y_i, t_i, c), \qquad t_i = i \Delta t,$$

from $i=0$ to $N-1$. Given $z_0$, the numerical trajectory is deterministic, and $y_N$ is an approximate target draw with error from field estimation, optimization, and integration. Prediction diversity comes from sampling different $z_0$ values, so the model is a deterministic conditional transport map driven by a random input.

Flow matching is attractive when its learned characteristics can be integrated accurately at low cost. For single-evaluation Euler, NFE equals the number of solver steps; multistage methods such as Heun use more than one field evaluation per accepted step. The optional reflow procedure pairs base samples with endpoints generated by a trained rectified-flow model and retrains on that model-induced coupling; repeated reflow can produce increasingly straight paths and improve coarse-step integration (Liu et al., 2023), but it does not guarantee a monotone reduction in every curvature or intersection measure. There is likewise no general per-round halving rule or portable step count. Required NFE depends on the learned field, coupling, solver, tolerance, model error, and task.

Consistency models (Song et al., 2023) train outputs to agree along a probability-flow trajectory, supporting one-step or few-step generation. Progressive distillation (Salimans and Ho, 2022) repeatedly trains a student to replace two deterministic diffusion-sampler steps with one. These methods reduce evaluation count in their demonstrated settings, but real-time control still requires wall-clock measurements on the target model and hardware.

The formalisms are related but not interchangeable. Under the regularity assumptions of the continuous-time diffusion-SDE formulation, a diffusion SDE admits a probability-flow ODE with the same ideal one-time marginals, and Flow Matching can train vector fields for diffusion probability paths. That equivalence does not make finite-capacity networks, objectives, pathwise trajectories, or discretized samplers identical. In practical flow models, path, coupling, parameterization, and architecture can affect training behavior, learned-trajectory curvature, and discretized sampling quality; these effects must be measured on the target benchmark. Rectification targets straighter characteristics, while distillation performance is separately method- and benchmark-dependent.