Score Matching is a training method for energy-based models that avoids computing the intractable partition function — by matching the gradient (score) of the model's log-density to the gradient of the data distribution, which does not require normalization.

How Score Matching Works

• Score: The score function is $s_ heta(x) = \nabla_x log p_ heta(x) = -\nabla_x E_ heta(x)$ (gradient of energy).
• Objective: Minimize $mathbb{E}_{p_{data}}[|s_ heta(x) - \nabla_x log p_{data}(x)|^2]$.
• Integration by Parts: The unknown $\nabla_x log p_{data}$ can be eliminated, giving: $mathbb{E}_{p_{data}}[ ext{tr}(\nabla_x s_ heta) + frac{1}{2}|s_ heta|^2]$.
• Denoising Score Matching: An equivalent objective that matches the score of the noise-perturbed distribution.

Why It Matters

• No Partition Function: Score matching completely avoids the intractable normalization problem.
• Diffusion Models: Modern diffusion models (DDPM, SDE-based) are trained with denoising score matching.
• Theoretically Sound: Score matching is consistent — the optimal model has the correct data score.

Score Matching is learning gradients instead of densities — training EBMs by matching the direction of steepest probability increase without computing $Z$.