Score-Based Generative Models are a class of generative models that learn the score function ∇_x log p(x)—the gradient of the log-probability density with respect to the data—rather than the density itself, then use the learned score to generate samples through iterative score-based sampling procedures such as Langevin dynamics. This approach avoids the normalization constant computation that makes direct density modeling intractable for complex, high-dimensional distributions.

Why Score-Based Generative Models Matter in AI/ML: Score-based models provide state-of-the-art generative quality by sidestepping the fundamental challenge of normalizing constant computation, leveraging the fact that the score function contains all the information needed for sampling without requiring a tractable partition function.

• Score function — The score ∇_x log p(x) is a vector field pointing in the direction of increasing log-density at every point in data space; following this gradient (with noise) from any starting point converges to samples from p(x) via Langevin dynamics • Score matching training — Directly minimizing E[||s_θ(x) - ∇_x log p(x)||²] is intractable (requires knowing the true score); denoising score matching instead trains on noisy data: s_θ(x̃) ≈ ∇_{x̃} log p(x̃|x) = -(x̃-x)/σ², which is tractable and consistent • Multi-scale noise perturbation — Score estimation is inaccurate in low-density regions (few training examples); adding noise at multiple scales (σ₁ > σ₂ > ... > σ_N) fills in low-density regions and creates a sequence of score functions from coarse to fine • Connection to diffusion — Score-based models and denoising diffusion probabilistic models (DDPMs) are equivalent formulations: the DDPM denoiser ε_θ is related to the score by s_θ(x_t, t) = -ε_θ(x_t, t)/σ_t; this unification bridges the two research communities • SDE formulation — Song et al. unified score-based and diffusion models through stochastic differential equations (SDEs): the forward SDE gradually adds noise, and the reverse-time SDE (requiring the score function) generates samples by denoising

Score-based generative models represent a paradigm shift in generative modeling by learning the gradient of the log-density rather than the density itself, unifying with diffusion models through the SDE framework and achieving state-of-the-art image generation quality by sidestepping normalization constant computation while enabling flexible, iterative sampling through learned score functions.