Autoregressive Flows are a class of normalizing flow models that construct invertible transformations using autoregressive structure, where each output dimension depends only on the previous dimensions through a triangular Jacobian matrix. This autoregressive constraint enables exact and efficient computation of both the forward transformation and its log-determinant Jacobian, making density evaluation and sampling tractable while maintaining the expressiveness to model complex distributions.

Why Autoregressive Flows Matter in AI/ML: Autoregressive flows provide exact density evaluation with flexible, learnable transformations, enabling precise likelihood computation for generative modeling, variational inference, and density estimation tasks where approximate methods are insufficient.

• Triangular Jacobian — The autoregressive structure produces a lower-triangular Jacobian matrix whose determinant is simply the product of diagonal elements: log|det J| = Σ log|∂y_i/∂x_i|; this O(d) computation replaces the general O(d³) determinant, making flows practical for high dimensions • Masked Autoregressive Flow (MAF) — Each layer transforms x_i → y_i = x_i · exp(s_i(x_{Inverse Autoregressive Flow (IAF) — Reverses MAF's computational tradeoff: y_i = x_i · exp(s_i(y_{Affine coupling layers — A simpler variant (RealNVP) splits dimensions into two groups: y_{1:d} = x_{1:d} (identity) and y_{d+1:D} = x_{d+1:D} · exp(s(x_{1:d})) + t(x_{1:d}); both forward and inverse are O(1) but less expressive per layer • Stacking for expressiveness — Individual autoregressive layers have limited expressiveness; stacking multiple layers with permuted dimension orderings builds complex, multi-modal distributions while maintaining exact invertibility and tractable Jacobians

Autoregressive flows are the foundational architecture for normalizing flow models, providing exact, tractable density evaluation through the elegant constraint of triangular Jacobian structure that transforms complex distribution modeling into a sequence of simple, invertible transformations composable into arbitrarily expressive density estimators.