diffusion model denoising

**Diffusion Models** are **generative models that learn to reverse a gradual noise-addition process, training a neural network to predict and remove noise at each step — generating high-quality images, audio, and video by iteratively denoising random Gaussian noise into structured data through a learned reverse process**. **Forward Process (Noise Addition):** - **Gaussian Noise Schedule**: given data sample x₀, gradually add Gaussian noise over T timesteps (T=1000 typically); at timestep t, x_t = √ᾱ_t · x₀ + √(1-ᾱ_t) · ε where ε ~ N(0,I) and ᾱ_t decreases from 1 to ~0; the forward process is fixed (not learned), only the reverse is trained - **Noise Schedule Design**: linear schedule (β_t from 0.0001 to 0.02) was original DDPM; cosine schedule provides more gradual corruption in early steps, preserving image structure longer and improving sample quality; VP (variance-preserving) vs VE (variance-exploding) formulations provide different mathematical treatments - **Signal-to-Noise Ratio**: SNR(t) = ᾱ_t / (1-ᾱ_t) decreases monotonically; early timesteps (high SNR) capture global structure; late timesteps (low SNR) capture fine details; training loss can be weighted by SNR to emphasize different generation aspects - **Continuous Time**: discrete timesteps T→∞ converges to a stochastic differential equation (SDE); enables theoretical analysis through SDE/ODE solvers and provides a unified framework for score-based and DDPM models **Reverse Process (Denoising):** - **Noise Prediction**: neural network ε_θ(x_t, t) predicts the noise ε added at timestep t; equivalently, predicts the score function ∇_x log p(x_t) — both formulations are mathematically equivalent and lead to the same training objective - **Training Objective**: minimize E[||ε - ε_θ(x_t, t)||²] — simple mean squared error between predicted and actual noise; this denoising score matching objective is remarkably simple yet produces state-of-the-art generative models - **Architecture (U-Net)**: standard DDPM uses a U-Net with residual blocks, spatial attention, and timestep conditioning (via sinusoidal embeddings + FiLM conditioning); downsampling/upsampling path with skip connections captures multi-scale features - **Conditioning**: text conditioning via cross-attention (inject CLIP text embeddings into U-Net attention layers); classifier-free guidance (CFG) trains with conditional and unconditional objectives, interpolating at inference: ε_guided = ε_uncond + w·(ε_cond - ε_uncond) with guidance scale w=7-15 **Sampling Acceleration:** - **DDIM (Denoising Diffusion Implicit Models)**: deterministic sampling using non-Markovian reverse process; skips timesteps (1000→50 steps) with minimal quality loss; enables interpolation in latent space and deterministic generation from fixed noise - **DPM-Solver**: high-order ODE solver (2nd/3rd order) for the probability flow ODE; achieves high-quality samples in 10-25 steps — 40-100× faster than original 1000-step DDPM - **Distillation**: progressive distillation (Salimans & Ho 2022) trains student to match teacher's two-step output in one step; repeatedly halving steps achieves 4-8 step generation; consistency models (Song et al. 2023) enable single-step generation **Latent Diffusion (Stable Diffusion):** - **Architecture**: encodes images to a compressed latent space via VAE (8× spatial compression); diffusion operates in latent space rather than pixel space — 64× less computation than pixel-space diffusion - **Components**: VAE encoder/decoder + U-Net denoiser + CLIP text encoder; modular design enables swapping components (different VAEs, different text encoders, custom U-Nets) - **ControlNet**: auxiliary networks that add spatial conditioning (edges, poses, depth maps) to pre-trained diffusion models without modifying the base model; enables precise compositional control - **SDXL/SD3**: SDXL adds second text encoder and refiner network; SD3 replaces U-Net with DiT (Diffusion Transformer) backbone achieving better text-image alignment and composition Diffusion models are **the dominant generative paradigm of the 2020s — their mathematical elegance, training stability, and unprecedented output quality have displaced GANs in image generation and enabled revolutionary applications in text-to-image, video generation, molecular design, and protein structure prediction**.