supermasks

**Supermasks** are a **binary mask applied to a randomly initialized neural network that achieves good performance without any weight training** — demonstrating that a sufficiently overparameterized random network already contains useful sub-networks. **What Is a Supermask?** - **Concept**: Instead of learning weights, learn which weights to keep (binary mask optimization). - **Process**: Fix weights at random init $ heta_0$. Optimize mask $m in {0,1}^n$. Inference: $m odot heta_0$. - **Finding**: A random dense network + learned mask can achieve ~95% of trained network accuracy on MNIST. **Why It Matters** - **Extreme Efficiency**: Only 1 bit per parameter (on/off) needs to be learned, not 32-bit floats. - **Theory**: Supports the "Strong Lottery Ticket" hypothesis — that random networks contain solutions without training. - **Hardware**: Could enable ultra-low-power inference with fixed random weights and binary masks. **Supermasks** are **finding intelligence in randomness** — proving that the structure of connections matters more than the values of the weights.