Score-Based Generative Models via SDEs are a theoretical unification of score matching and diffusion models through the framework of stochastic differential equations — showing that both approaches instantiate a general pattern: a forward SDE continuously transforms data into noise while a reverse SDE (conditioned on the learned score function ∇log p_t(x)) transforms noise back into data, enabling flexible noise schedules, exact likelihood computation via a probability flow ODE, and controllable generation that subsumed all prior score matching and DDPM methods into a single mathematical framework.

The Unifying Forward SDE

The forward process transforms data x₀ into noise through a continuous SDE:

dx = f(x, t) dt + g(t) dW

where:

• f(x, t): drift coefficient (determines deterministic flow)
• g(t): diffusion coefficient (controls noise injection rate)
• W: standard Wiener process (Brownian motion)

Different choices of f and g recover all prior methods:

All converge to a tractable noise distribution (Gaussian) at t=T, from which sampling is trivial.

The Reverse SDE: Denoising as Time Reversal

Anderson (1982) showed that any forward diffusion SDE has an exact reverse-time SDE:

dx = [f(x, t) - g²(t) ∇_x log p_t(x)] dt + g(t) dW̄

where dW̄ is reverse-time Brownian motion and ∇_x log p_t(x) is the score function — the gradient of the log probability density with respect to the data at noise level t.

The score function is the critical quantity. It is unknown analytically but can be learned by a neural network s_θ(x, t) ≈ ∇_x log p_t(x) via denoising score matching:

L(θ) = E_{t, x₀, ε}[||s_θ(x_t, t) - ∇_{x_t} log p(x_t | x₀)||²]

= E_{t, x₀, ε}[||s_θ(x₀ + σ_t ε, t) + ε/σ_t||²]

This is exactly the denoising objective used in DDPM — demonstrating that DDPM implicitly learns the score function.

Sampling Methods

Once the score network s_θ is trained, multiple sampling algorithms apply:

Langevin MCMC (discrete steps): x_{n+1} = x_n + ε ∇_x log p(x_n) + √(2ε) z, iterating from pure noise at decreasing noise levels (annealed Langevin dynamics).

Reverse SDE (stochastic): Simulate the reverse SDE using Euler-Maruyama or Predictor-Corrector methods. Produces diverse samples with good coverage of the data distribution.

Probability Flow ODE (deterministic): The corresponding ODE whose marginals match the SDE at every t:

dx/dt = f(x, t) - ½ g²(t) ∇_x log p_t(x)

This ODE has identical marginal distributions to the reverse SDE but is deterministic — enabling:

• Exact likelihood computation via the instantaneous change-of-variables formula (without volume-preserving constraints of normalizing flows)
• Deterministic interpolation between data points in latent space
• Faster sampling using high-order ODE solvers (DDIM, DPM-Solver)

Controllable Generation

The score function framework enables controlled generation without retraining:

Classifier guidance: ∇_x log p_t(x|y) = ∇_x log p_t(x) + ∇_x log p_t(y|x)

Train a noisy classifier p_t(y|x) and add its gradient to the score function. The combined score pushes samples toward class y.

Classifier-free guidance: Learn conditional and unconditional score jointly, interpolate at sampling time: s_guided = s_unconditional + w × (s_conditional - s_unconditional). This approach — used in Stable Diffusion — avoids the noisy classifier and typically produces higher-quality samples.

Impact and Legacy

This SDE framework, introduced by Song et al. (2020), unified the fragmented literature connecting SMLD (Noise Conditional Score Networks), DDPM, and score matching into a single principled theory. It enabled:

• Stable Diffusion (VP-SDE backbone)
• DALL-E 2 (DDPM with CLIP guidance)
• Theoretical analysis of diffusion model convergence
• DPM-Solver and other fast samplers derived from ODE analysis

The probability flow ODE connection transformed diffusion models from "interesting generative models" into a theoretically complete framework with exact likelihoods — equivalent in expressive power to normalizing flows but without their architectural constraints.