Diffusion Models are generative models that learn to reverse a gradual noising process, transforming pure Gaussian noise back into structured data through iterative denoising steps — producing state-of-the-art image, audio, and video generation quality that has surpassed GANs, powering systems like Stable Diffusion, DALL-E 3, Midjourney, and Sora.

Forward Process (Adding Noise)

• Start with a clean data sample x₀ (e.g., an image).
• At each timestep t, add a small amount of Gaussian noise: $x_t = \sqrt{\alpha_t} \cdot x_{t-1} + \sqrt{1 - \alpha_t} \cdot \epsilon$.
• After T steps (T ≈ 1000): x_T ≈ pure Gaussian noise.
• This process requires no learning — it's a fixed schedule.

Reverse Process (Denoising — The Learned Part)

• A neural network (typically a U-Net or Transformer) learns to predict the noise ε added at each step.
• Starting from pure noise x_T, iteratively denoise: $x_{t-1} = \frac{1}{\sqrt{\alpha_t}}(x_t - \frac{1-\alpha_t}{\sqrt{1-\bar{\alpha}_t}} \epsilon_\theta(x_t, t)) + \sigma_t z$.
• After T reverse steps → generates a clean sample from the learned distribution.

Training Objective

• Simple MSE loss: $L = E_{t, x_0, \epsilon}[||\epsilon - \epsilon_\theta(x_t, t)||^2]$.
• Sample a random timestep t, add noise to get x_t, predict the noise, minimize error.
• No adversarial training, no mode collapse — stable optimization.

Key Variants

Conditioning and Guidance

• Text conditioning: Text encoder (CLIP/T5) provides embedding → cross-attention in U-Net.
• Classifier-Free Guidance (CFG): $\epsilon_{guided} = \epsilon_{uncond} + w \cdot (\epsilon_{cond} - \epsilon_{uncond})$.
• Scale w = 7-15 for high-quality, text-aligned generation.
• ControlNet: Additional conditioning on edges, depth maps, poses.

Latent Diffusion (Stable Diffusion Architecture)

• VAE encodes 512×512 image → 64×64 latent representation (8x compression).
• Diffusion operates in latent space → 64x less computation than pixel-space diffusion.
• U-Net with cross-attention for text conditioning.
• VAE decoder converts denoised latent back to pixel image.

Diffusion models are the dominant generative paradigm as of 2024-2025 — their combination of training stability, output quality, and flexible conditioning has made them the foundation of commercial image generation, video synthesis, drug design, and audio generation systems.