Diffusion Models are generative models that learn to reverse a gradual noise-addition process, training a neural network to predict and remove noise at each step — generating high-quality images, audio, and video by iteratively denoising random Gaussian noise into structured data through a learned reverse process.

Forward Process (Noise Addition):

• Gaussian Noise Schedule: given data sample x₀, gradually add Gaussian noise over T timesteps (T=1000 typically); at timestep t, x_t = √ᾱ_t · x₀ + √(1-ᾱ_t) · ε where ε ~ N(0,I) and ᾱ_t decreases from 1 to ~0; the forward process is fixed (not learned), only the reverse is trained
• Noise Schedule Design: linear schedule (β_t from 0.0001 to 0.02) was original DDPM; cosine schedule provides more gradual corruption in early steps, preserving image structure longer and improving sample quality; VP (variance-preserving) vs VE (variance-exploding) formulations provide different mathematical treatments
• Signal-to-Noise Ratio: SNR(t) = ᾱ_t / (1-ᾱ_t) decreases monotonically; early timesteps (high SNR) capture global structure; late timesteps (low SNR) capture fine details; training loss can be weighted by SNR to emphasize different generation aspects
• Continuous Time: discrete timesteps T→∞ converges to a stochastic differential equation (SDE); enables theoretical analysis through SDE/ODE solvers and provides a unified framework for score-based and DDPM models

Reverse Process (Denoising):

• Noise Prediction: neural network ε_θ(x_t, t) predicts the noise ε added at timestep t; equivalently, predicts the score function ∇_x log p(x_t) — both formulations are mathematically equivalent and lead to the same training objective
• Training Objective: minimize E[||ε - ε_θ(x_t, t)||²] — simple mean squared error between predicted and actual noise; this denoising score matching objective is remarkably simple yet produces state-of-the-art generative models
• Architecture (U-Net): standard DDPM uses a U-Net with residual blocks, spatial attention, and timestep conditioning (via sinusoidal embeddings + FiLM conditioning); downsampling/upsampling path with skip connections captures multi-scale features
• Conditioning: text conditioning via cross-attention (inject CLIP text embeddings into U-Net attention layers); classifier-free guidance (CFG) trains with conditional and unconditional objectives, interpolating at inference: ε_guided = ε_uncond + w·(ε_cond - ε_uncond) with guidance scale w=7-15

Sampling Acceleration:

• DDIM (Denoising Diffusion Implicit Models): deterministic sampling using non-Markovian reverse process; skips timesteps (1000→50 steps) with minimal quality loss; enables interpolation in latent space and deterministic generation from fixed noise
• DPM-Solver: high-order ODE solver (2nd/3rd order) for the probability flow ODE; achieves high-quality samples in 10-25 steps — 40-100× faster than original 1000-step DDPM
• Distillation: progressive distillation (Salimans & Ho 2022) trains student to match teacher's two-step output in one step; repeatedly halving steps achieves 4-8 step generation; consistency models (Song et al. 2023) enable single-step generation

Latent Diffusion (Stable Diffusion):

• Architecture: encodes images to a compressed latent space via VAE (8× spatial compression); diffusion operates in latent space rather than pixel space — 64× less computation than pixel-space diffusion
• Components: VAE encoder/decoder + U-Net denoiser + CLIP text encoder; modular design enables swapping components (different VAEs, different text encoders, custom U-Nets)
• ControlNet: auxiliary networks that add spatial conditioning (edges, poses, depth maps) to pre-trained diffusion models without modifying the base model; enables precise compositional control
• SDXL/SD3: SDXL adds second text encoder and refiner network; SD3 replaces U-Net with DiT (Diffusion Transformer) backbone achieving better text-image alignment and composition

Diffusion models are the dominant generative paradigm of the 2020s — their mathematical elegance, training stability, and unprecedented output quality have displaced GANs in image generation and enabled revolutionary applications in text-to-image, video generation, molecular design, and protein structure prediction.