weight entanglement

**Weight Entanglement** is a **phenomenon in weight-sharing NAS methods where the shared weights of sub-networks interfere with each other** — preventing accurate performance estimation because training one sub-network path affects the weights used by other paths. **What Is Weight Entanglement?** - **Problem**: In one-shot NAS (like DARTS), all sub-networks share the same set of weights. Training improves one sub-network but may degrade others. - **Consequence**: The ranking of sub-architectures using shared weights does not match their ranking when trained independently. - **Severity**: More severe with larger search spaces and more shared paths. **Why It Matters** - **NAS Reliability**: Weight entanglement is the primary reason one-shot NAS methods sometimes find sub-optimal architectures. - **Solutions**: Progressive shrinking (OFA), few-shot NAS (split into multiple sub-supernets), or training longer to reduce interference. - **Research**: Understanding and mitigating weight entanglement is an active area of NAS research. **Weight Entanglement** is **the interference pattern in shared-weight NAS** — where training one architecture pathway inadvertently disrupts the performance of other pathways.