Multi-Exit Networks are neural networks designed with multiple output points throughout the architecture — each exit is a complete classifier, and the network can produce predictions at any exit point, enabling flexible accuracy-latency trade-offs at inference time.

Multi-Exit Design

• Exit Architecture: Each exit has its own pooling, feature transform, and classification head.
• Self-Distillation: Later exits teach earlier exits through knowledge distillation — improves early exit quality.
• Training Strategies: Weighted sum of all exit losses, curriculum learning, or gradient equilibrium.
• Orchestration: At inference, choose the exit based on input difficulty, latency budget, or confidence threshold.

Why It Matters

• Anytime Prediction: Can produce a prediction at any time — interrupted computation still gives a result.
• Device Adaptation: Same model serves different devices — powerful devices use all exits, weak devices exit early.
• Efficiency Scaling: Linear relationship between exits used and compute — predictable resource usage.

Multi-Exit Networks are the Swiss Army knife of inference — offering multiple accuracy-efficiency operating points within a single model.