normalizing flow generative

**Normalizing Flows** are the **generative model family that learns an invertible transformation between a simple base distribution (e.g., standard Gaussian) and a complex target distribution (e.g., natural images) — where the invertibility enables exact likelihood computation via the change-of-variables formula, and the transformation is composed of learnable invertible layers (coupling layers, autoregressive transforms, continuous flows) that progressively reshape the simple distribution into the complex data distribution**. **Mathematical Foundation** If z ~ p_z(z) is the base distribution and x = f(z) is the invertible transformation, the data distribution is: p_x(x) = p_z(f⁻¹(x)) × |det(∂f⁻¹/∂x)| The Jacobian determinant accounts for how the transformation stretches or compresses probability density. For the transformation to be practical: 1. f must be invertible (bijective). 2. The Jacobian determinant must be efficient to compute (not O(D³) for D-dimensional data). **Coupling Layer Architectures** **RealNVP / Glow**: - Split input into two halves: x = [x_a, x_b]. - Transform: y_a = x_a (identity), y_b = x_b ⊙ exp(s(x_a)) + t(x_a). - s() and t() are arbitrary neural networks (no invertibility requirement — they parameterize the transform, not perform it). - Jacobian is triangular → determinant is the product of diagonal elements (O(D) instead of O(D³)). - Inverse: x_b = (y_b - t(x_a)) ⊙ exp(-s(x_a)), x_a = y_a. Exact inversion! - Stack multiple coupling layers, alternating which half is transformed. **Autoregressive Flows (MAF, IAF)**: - Transform each dimension conditioned on all previous dimensions: x_i = z_i × exp(s_i(x_{