Efficient Neural Architecture Search (ENAS) is a neural architecture search method that reduces the computational cost of finding optimal network architectures from thousands of GPU-days to less than a single GPU-day by sharing weights across all candidate architectures in a search space — training one massive supergraph simultaneously and evaluating architectures by sampling subgraphs that inherit weights rather than training each candidate from scratch — introduced by Pham et al. (Google Brain, 2018) as the breakthrough that democratized NAS from a technique requiring industrial compute budgets to one feasible on a single GPU, enabling the broader community to explore automated architecture design.

What Is ENAS?

• Search Space as a DAG: ENAS represents the architecture search space as a directed acyclic graph (DAG) where each node represents a computation (layer) and each directed edge represents data flow. A particular path through this DAG is a candidate architecture.
• Weight Sharing: All candidate architectures within the DAG share a single set of parameters — the weights of the supergraph. When a specific architecture is sampled and evaluated, its layers use the corresponding subgraph's weights directly, without retraining.
• Controller (RNN): A recurrent neural network serves as the architecture controller — at each step, the RNN decides which edges and operations to include in the child architecture by sampling from categorical distributions.
• RL Training of Controller: The controller is trained with reinforcement learning, rewarded by the validation accuracy of the architectures it samples (evaluated using shared weights — fast inference rather than full training).
• Two Optimization Loops: (1) Train shared weights with gradient descent (update supergraph to support all sampled architectures); (2) Train the controller with REINFORCE to select better architectures.

Why ENAS Is Revolutionary

• Cost Reduction: Original NAS (Zoph & Le, 2017) required 450 GPU-days and 800 GPU workers. ENAS reduces this to 0.45 GPU-days — a 1,000× speedup.
• Amortization: Training cost is amortized across the entire search space — weight sharing means every architecture benefits from every gradient step taken anywhere in the supergraph.
• Democratization: ENAS made NAS accessible to academic labs with a single GPU, spawning hundreds of follow-up works exploring diverse search spaces, tasks, and domains.
• Iterative Refinement: The controller can quickly sample and evaluate thousands of architectures per hour, exploring the search space far more thoroughly than random search.

Weight Sharing: Trade-offs and Challenges

The ranking correlation issue — whether architectures ranked well by shared weights are also ranked well after standalone training — is a central research question addressed by follow-up work including SNAS, DARTS, and One-Shot NAS.

Influence on NAS Research

• DARTS: Replaced discrete architecture sampling with continuous relaxation — differentiable architecture search in the supergraph.
• Once-for-All (OFA): Extended weight sharing to produce a single network that, without retraining, can be sliced to different widths/depths for different hardware targets.
• ProxylessNAS: Direct search on target hardware (mobile devices) using ENAS-style weight sharing with hardware-aware latency objectives.
• AutoML: ENAS is the foundation of automated model design pipelines used in production at Google, Meta, and Huawei.

ENAS is the NAS breakthrough that made automated architecture design practical — proving that sharing weights across an entire search space enables exploration of millions of candidate architectures at the cost of training just one, transforming neural architecture search from a billionaire's toy into an everyday research tool.