binarized neural networks (bnn)

**Binarized Neural Networks (BNN)** are a **specific implementation framework for training and deploying binary neural networks** — using the Straight-Through Estimator (STE) to handle the non-differentiable sign function during backpropagation. **What Is a BNN?** - **Forward Pass**: Binarize weights and activations using the sign function ($+1$ if $x geq 0$, else $-1$). - **Backward Pass**: The sign function has zero gradient almost everywhere. The STE uses the gradient of a smooth approximation (hard tanh or identity) instead. - **Latent Weights**: Full-precision "shadow" weights are maintained for gradient accumulation, then binarized for the forward pass. **Why It Matters** - **Pioneering**: Courbariaux et al. (2016) demonstrated the first practical BNN training procedure. - **Foundation**: All subsequent binary/ternary network methods build on the STE trick introduced here. - **FPGA Deployment**: BNNs are the go-to architecture for FPGA-based inference accelerators. **Binarized Neural Networks** are **the engineering blueprint for 1-bit AI** — solving the fundamental training challenge of discrete-valued networks.