model merging
**Model Merging** is the **technique of combining the weights of multiple independently fine-tuned models into a single model without additional training** — creating models that inherit capabilities from all parent models simultaneously, enabling zero-cost composition of specialized skills like coding + instruction following + math reasoning into one unified model.
**Why Model Merging?**
- Fine-tune separate models for: coding, math, creative writing, medical knowledge.
- Merging combines all skills into one model — no multi-task training data needed.
- Zero additional compute: Just arithmetic on weight tensors.
- Community innovation: Merged models frequently top open-source leaderboards.
**Merging Methods**
| Method | Technique | Strengths |
|--------|----------|----------|
| Linear (Lerp) | $W = (1-\alpha)W_A + \alpha W_B$ | Simple, effective baseline |
| SLERP | Spherical interpolation | Preserves weight magnitudes better |
| TIES | Trim, Elect Sign, Merge | Resolves parameter conflicts |
| DARE | Drop And REscale | Randomly drops delta params before merge |
| Task Arithmetic | Add task vectors to base | Compositional task addition |
| Model Soups | Average multiple fine-tuned models | Robust, reduces variance |
**SLERP (Spherical Linear Interpolation)**
$W = \frac{\sin((1-t)\Omega)}{\sin(\Omega)} W_A + \frac{\sin(t\Omega)}{\sin(\Omega)} W_B$
where $\Omega = \arccos(\frac{W_A \cdot W_B}{||W_A|| \cdot ||W_B||})$
- Interpolates along the great circle on the unit hypersphere.
- Better than linear interpolation for preserving the geometry of weight space.
- Only works for 2 models — iterative application needed for 3+.
**TIES-Merging (Yadav et al., 2023)**
1. **Trim**: Zero out small-magnitude task vector components (keep top-K%).
2. **Elect Sign**: For each parameter, use majority sign across models (resolve conflicts).
3. **Merge**: Average the remaining aligned parameters.
- Addresses the interference problem: Different fine-tunes may push same parameter in opposite directions.
**DARE (Yu et al., 2023)**
1. Compute task vectors: $\Delta W_i = W_{fine-tuned,i} - W_{base}$.
2. Randomly drop (set to zero) p% of delta parameters (p=90-99%).
3. Rescale remaining: $\Delta W_i' = \Delta W_i / (1-p)$.
4. Merge rescaled deltas.
- Key insight: Most fine-tuning changes are redundant — only a few are critical.
**Task Arithmetic**
$W_{merged} = W_{base} + \lambda_1 \tau_1 + \lambda_2 \tau_2 + ...$
where $\tau_i = W_{fine-tuned,i} - W_{base}$ (task vector)
- Can also **negate** task vectors: Subtract a toxicity task vector → less toxic model.
- λ controls strength of each task (typically 0.5-1.5).
**Practical Tips**
- Models must share the same base model (e.g., all fine-tuned from LLaMA-3-8B).
- SLERP: Best for merging 2 models with complementary skills.
- DARE + TIES: Best for merging 3+ models.
- Always evaluate merged model — not all combinations produce improvements.
**Tools**: mergekit (most popular), Hugging Face model merger, LM-Cocktail.
Model merging is **a uniquely practical innovation from the open-source AI community** — by enabling zero-cost combination of specialized capabilities, it has become the dominant technique for creating top-performing open-source models and represents a form of collective intelligence where independent fine-tuning efforts compound.