denoising diffusion probabilistic models (ddpm)

Denoising Diffusion Probabilistic Models (DDPMs) provide the core mathematical framework for diffusion-based generative models, learning to reverse a gradual noising process to generate high-quality samples from pure noise. The framework defines two processes: the forward (diffusion) process, which incrementally adds Gaussian noise to data over T timesteps according to a fixed variance schedule β₁, β₂, ..., β_T (q(x_t|x_{t-1}) = N(x_t; √(1-β_t) x_{t-1}, β_t I)), and the reverse (denoising) process, which learns to remove noise step by step (p_θ(x_{t-1}|x_t) = N(x_{t-1}; μ_θ(x_t, t), σ_t² I)). The forward process has a closed-form solution: x_t = √(ᾱ_t) x_0 + √(1-ᾱ_t) ε, where ᾱ_t is the cumulative product of (1-β_t) terms and ε ~ N(0,I). This allows sampling any noisy version x_t directly without iterating through intermediate steps. The neural network (typically a U-Net with attention layers and time-step embeddings) is trained to predict the noise ε added at each timestep, with the simplified training objective: L = E[||ε - ε_θ(x_t, t)||²]. At generation time, starting from pure Gaussian noise x_T, the model iteratively denoises: predict the noise component, subtract it (with appropriate scaling), and add a small amount of fresh noise (the stochastic sampling step). Key innovations from the seminal Ho et al. (2020) paper include the simplified training objective, the reparameterization to predict noise rather than the mean, and demonstrating that diffusion models can match or exceed GANs in image quality. DDPMs spawned numerous improvements: DDIM (deterministic sampling enabling fewer steps), classifier-free guidance (trading diversity for quality), latent diffusion (operating in compressed latent space for efficiency), and score-based formulations connecting to stochastic differential equations.