layer skipping
**Layer Skipping** is a **transformer inference optimization technique that bypasses intermediate layers for tokens or sequences that do not require full-depth processing, using learned skip connections, router-based decisions, or progressive training strategies that build skip-robust representations** — exploiting the empirical observation that many transformer layers perform incremental refinements rather than critical transformations, and that later layers often contribute marginally for straightforward inputs.
**What Is Layer Skipping?**
- **Definition**: Layer skipping modifies the standard sequential layer-by-layer processing of transformers by allowing tokens to jump directly from layer N to layer N+K via the residual connection, bypassing the self-attention and feed-forward computation of the intervening layers. The decision of which layers to skip can be static (predetermined), learned (router-based), or stochastic (random during training for robustness).
- **Residual Bypass**: The skip mechanism leverages the residual connections already present in transformer architectures. When a layer is skipped, the token's hidden state passes unchanged through the residual stream to the next active layer — meaning skipping is computationally free and does not require special architectural modifications beyond the routing decision.
- **Distinction from Early Exit**: Early exit terminates all computation at an intermediate layer and produces a final output. Layer skipping selectively bypasses specific layers while continuing processing at deeper layers — allowing the network to access the final layers' representations even when intermediate layers are bypassed.
**Why Layer Skipping Matters**
- **Inference Speedup**: Bypassing 20–40% of layers reduces inference FLOP count proportionally. For autoregressive generation where the forward pass is the bottleneck, this translates directly to tokens-per-second improvement. Implementations report 20–40% latency reduction with less than 1% quality degradation on standard benchmarks.
- **Layer Redundancy**: Empirical analysis of trained transformers reveals significant redundancy in intermediate layers. CKA (Centered Kernel Alignment) similarity between consecutive layer representations is often >0.95, indicating that adjacent layers make only minor refinements. Layer skipping exploits this redundancy by bypassing near-duplicate layers.
- **Training Robustness**: Progressive layer dropping during training (randomly skipping layers with increasing probability) forces the network to build representations that are robust to missing intermediate computation. This creates a model that can tolerate layer skipping at inference without the quality collapse that would occur in a conventionally trained model.
- **Complementary to Quantization**: Layer skipping and weight quantization are orthogonal optimization axes that can be combined. A model with 50% layer skip and 4-bit quantization achieves compound efficiency gains — reducing both arithmetic intensity (fewer layers) and memory bandwidth (smaller weights per layer).
**Layer Skipping Approaches**
| Technique | Mechanism | Key Benefit |
|-----------|-----------|-------------|
| **Stochastic Depth** | Random layer dropping during training | Builds skip-robust representations |
| **Learned Routing** | Per-token router decides skip/execute at each layer | Adaptive to input difficulty |
| **Static Pruning** | Remove least-important layers post-training based on importance metrics | Simple deployment, no routing overhead |
| **Block Skipping** | Skip groups of consecutive layers rather than individual layers | Reduces routing decisions |
**Layer Skipping** is **selective depth processing** — the inference optimization that recognizes not every transformer layer contributes equally to every prediction, enabling models to bypass redundant computation while preserving the critical processing pathways that determine output quality.