diffusion model
**Diffusion Models** are **generative models that learn to reverse a gradual noising process, transforming pure Gaussian noise back into structured data through iterative denoising steps** — producing state-of-the-art image, audio, and video generation quality that has surpassed GANs, powering systems like Stable Diffusion, DALL-E 3, Midjourney, and Sora.
**Forward Process (Adding Noise)**
- Start with a clean data sample x₀ (e.g., an image).
- At each timestep t, add a small amount of Gaussian noise: $x_t = \sqrt{\alpha_t} \cdot x_{t-1} + \sqrt{1 - \alpha_t} \cdot \epsilon$.
- After T steps (T ≈ 1000): x_T ≈ pure Gaussian noise.
- This process requires no learning — it's a fixed schedule.
**Reverse Process (Denoising — The Learned Part)**
- A neural network (typically a U-Net or Transformer) learns to predict the noise ε added at each step.
- Starting from pure noise x_T, iteratively denoise: $x_{t-1} = \frac{1}{\sqrt{\alpha_t}}(x_t - \frac{1-\alpha_t}{\sqrt{1-\bar{\alpha}_t}} \epsilon_\theta(x_t, t)) + \sigma_t z$.
- After T reverse steps → generates a clean sample from the learned distribution.
**Training Objective**
- Simple MSE loss: $L = E_{t, x_0, \epsilon}[||\epsilon - \epsilon_\theta(x_t, t)||^2]$.
- Sample a random timestep t, add noise to get x_t, predict the noise, minimize error.
- No adversarial training, no mode collapse — stable optimization.
**Key Variants**
| Model | Innovation | Speed |
|-------|-----------|-------|
| DDPM (Ho et al. 2020) | Original formulation | Slow (1000 steps) |
| DDIM | Deterministic sampling, fewer steps | 10-50 steps |
| Latent Diffusion (LDM) | Diffuse in VAE latent space, not pixel space | Fast (Stable Diffusion) |
| Flow Matching | Straighter ODE paths | 1-10 steps possible |
| Consistency Models | Direct single-step generation | 1-2 steps |
**Conditioning and Guidance**
- **Text conditioning**: Text encoder (CLIP/T5) provides embedding → cross-attention in U-Net.
- **Classifier-Free Guidance (CFG)**: $\epsilon_{guided} = \epsilon_{uncond} + w \cdot (\epsilon_{cond} - \epsilon_{uncond})$.
- Scale w = 7-15 for high-quality, text-aligned generation.
- **ControlNet**: Additional conditioning on edges, depth maps, poses.
**Latent Diffusion (Stable Diffusion Architecture)**
- VAE encodes 512×512 image → 64×64 latent representation (8x compression).
- Diffusion operates in latent space → 64x less computation than pixel-space diffusion.
- U-Net with cross-attention for text conditioning.
- VAE decoder converts denoised latent back to pixel image.
Diffusion models are **the dominant generative paradigm as of 2024-2025** — their combination of training stability, output quality, and flexible conditioning has made them the foundation of commercial image generation, video synthesis, drug design, and audio generation systems.