Flow Matching for Generative Modeling

This article introduces Flow Matching as an efficient framework for generative modeling, focusing on its theoretical foundations and practical algorithms.

Diffusion models aim to sample complex high‑dimensional data (e.g., natural images) by transforming a simple distribution (usually Gaussian) into the target distribution. Traditional diffusion samplers require solving stochastic differential equations, which can be computationally intensive.

Flow Matching reformulates the problem as learning a time‑dependent vector field (a flow) that deterministically transports samples from the simple distribution to the data distribution. The flow satisfies a continuous normalizing flow (CNF) ordinary differential equation, and the mapping can be expressed analytically for simple cases such as two Gaussian distributions with identical variance.

For more complex distributions, the method treats them as mixtures of delta functions. By linearly combining the conditional vector fields of each pair of deltas, the expected flow at any time can be computed, providing a closed‑form target for training.

To avoid enumerating all possible pairs, the vector field is parameterized by a neural network. An L2 loss between the network output and the analytically derived conditional flow yields an optimal solution equal to the desired expectation. This loss is minimized by standard gradient descent.

The training algorithm proceeds as follows: (1) initialize the network; (2) repeatedly sample a data point and a random time t; (3) sample a Gaussian noise point at time t; (4) compute the target flow using the conditional formula; (5) compute the L2 loss between the network prediction and the target; (6) update the network parameters. The trained network directly provides the flow for any (x, t).

During inference, a sample is drawn from the Gaussian prior, and the learned flow is integrated forward in time (e.g., with Euler steps) to obtain a data sample. This deterministic integration replaces the stochastic denoising steps of DDPM.

Compared with classic diffusion models, Flow Matching offers a simpler training pipeline, deterministic sampling, and often faster convergence while achieving comparable or better generation quality.

The article concludes with a summary of the advantages of Flow Matching and lists several recent papers on the topic for further reading.