universally slimmable networks

**Universally Slimmable Networks (US-Nets)** are an **extension of slimmable networks that support any arbitrary width multiplier, not just preset values** — enabling continuous, fine-grained accuracy-efficiency trade-offs at runtime. **US-Net Training** - **Any Width**: US-Nets support any width from the minimum to maximum (e.g., any value between 0.25× and 1.0×). - **Sandwich Rule**: During training, always train the smallest and largest width (bread), plus $n$ random widths (filling). - **In-Place Distillation**: The largest width acts as teacher — its soft labels guide the smaller widths. - **Switchable BN**: Separate batch norm statistics for each width — essential for multi-width training. **Why It Matters** - **Infinite Configs**: Not limited to 4 preset widths — any width is available at runtime. - **Hardware Matching**: Exactly match any hardware's computation budget — not just the nearest preset. - **Smooth Degradation**: Performance degrades smoothly as width decreases — no sudden accuracy drops. **US-Nets** are **infinitely adjustable models** — supporting any width configuration for perfectly fine-grained accuracy-efficiency control.