model stitching

**Model Stitching** is a **technique that combines layers from different pre-trained models into a single network** — inserting a small "stitching layer" (typically a 1×1 convolution or linear layer) between layers from different models to align their representations. **How Does Model Stitching Work?** - **Source Models**: Two or more pre-trained models trained independently. - **Cut Points**: Select layer $i$ from model $A$ and layer $j$ from model $B$. - **Stitch**: Insert a trainable stitching layer between layer $i$ and layer $j$. - **Train Stitch**: Train only the stitching layer (freeze source model weights). - **Result**: Front of model $A$ + stitch + back of model $B$. **Why It Matters** - **Representation Analysis**: Reveals how similar representations are between different models at different layers. - **Efficiency**: Create models with novel accuracy-efficiency trade-offs by combining parts of different architectures. - **Transfer**: Transfer the "front end" of one model with the "back end" of another. **Model Stitching** is **Frankenstein assembly for neural networks** — combining parts of different models with minimal adaptation layers.