Once-for-All (OFA) is a NAS approach that trains a single large "supernet" that supports many sub-networks — enabling deployment of different-sized architectures for different hardware targets without re-training, by simply selecting the appropriate sub-network.

How Does OFA Work?

• Progressive Shrinking: Train the supernet with progressively smaller sub-networks (first full model, then reduced depth, then reduced width, then reduced kernel size and resolution).
• Elastic Dimensions: Supports variable depth (layer count), width (channel count), kernel size, and input resolution.
• Deployment: Given a hardware constraint, search for the best sub-network within the trained supernet.
• Paper: Cai et al. (2020).

Why It Matters

• Train Once: A single training run produces models for every deployment scenario (cloud, mobile, IoT, edge).
• Massive Efficiency: Eliminates re-training for each target -> 10-100x reduction in total NAS compute.
• Practical: Enables rapid customization of models for new hardware without ML expertise.

Once-for-All is the universal donor network — one model that contains optimized sub-networks for every possible deployment target.