one-shot nas

**One-Shot NAS** is a **weight-sharing NAS approach where a single "supernet" is trained that contains all candidate architectures as sub-networks** — enabling architecture evaluation without training each candidate from scratch, reducing search cost from thousands of GPU-hours to hours. **How Does One-Shot NAS Work?** - **Supernet**: A single overparameterized network containing all possible operations and connections. - **Training**: Train the supernet with random path sampling (at each iteration, activate a random sub-network). - **Evaluation**: To evaluate a candidate architecture, simply activate its corresponding paths in the trained supernet. No separate training needed. - **Search**: Use evolutionary search or RL to find the best sub-network within the trained supernet. **Why It Matters** - **Massive Speedup**: Train once, evaluate thousands of architectures by inheritance. - **Practical**: Makes NAS accessible on a single GPU (SPOS, OFA, FairNAS). - **Challenge**: Weight entanglement — shared weights may not accurately represent independently trained networks. **One-Shot NAS** is **all architectures in one network** — a clever weight-sharing trick that trades absolute accuracy for enormous search efficiency.