model merging
**Model Merging** is the **technique of combining the weights of multiple fine-tuned models into a single model without additional training** — by interpolating, averaging, or selectively combining parameters from models that share the same base architecture but were fine-tuned on different tasks or data, enabling multi-task capability, improved robustness, or novel capability combinations at zero additional training cost.
**Why Model Merging Works**
Fine-tuned models from the same pretrained base occupy a connected low-loss basin in the loss landscape. The key insight: linear interpolation between fine-tuned checkpoints often produces models that perform well on ALL constituent tasks, not just the average.
```
Base model → Fine-tune on Task A → Model_A (weights θ_A)
→ Fine-tune on Task B → Model_B (weights θ_B)
→ Fine-tune on Task C → Model_C (weights θ_C)
Merged: θ_merged = f(θ_A, θ_B, θ_C) → Good at A, B, AND C
```
**Merging Methods**
| Method | Formula | Key Idea |
|--------|---------|----------|
| Linear/SLERP | θ = α·θ_A + (1-α)·θ_B | Simple interpolation |
| Model Soup | θ = (1/N)·Σθ_i | Average multiple fine-tunes of same task |
| Task Arithmetic | θ = θ_base + Σλ_i·τ_i where τ_i = θ_i - θ_base | Add task vectors to base |
| TIES-Merging | Trim + Elect Sign + Merge | Resolve sign conflicts in task vectors |
| DARE | Random drop + rescale task vectors | Sparsify before merging |
| Frankenmerge | Layer-wise selection from different models | Pick best layers from each |
**Task Arithmetic**
The most influential framework defines a **task vector** τ = θ_fine-tuned - θ_base:
```python
# Task vectors capture what fine-tuning learned
task_vector_A = model_A.state_dict() - base.state_dict()
task_vector_B = model_B.state_dict() - base.state_dict()
# Addition: combine capabilities
merged = base + 0.7 * task_vector_A + 0.5 * task_vector_B
# Negation: remove capabilities (e.g., remove toxicity)
detoxified = base - 0.5 * task_vector_toxic
```
**TIES-Merging (Trim, Elect Sign, & Merge)**
Addresses interference when naively adding task vectors:
1. **Trim**: Zero out low-magnitude values (keep top-k% of each task vector)
2. **Elect Sign**: For each parameter, take majority vote on sign across task vectors
3. **Disjoint Merge**: Average only values that agree with the elected sign
**DARE (Drop And REscale)**
Randomly drops 90-99% of task vector values and rescales the rest — extremely sparse task vectors merge with less interference. Works especially well for LLMs where fine-tuning changes are highly redundant.
**Practical Applications**
- **Open-source LLM community**: Merging specialized LoRA adapters (code + chat + reasoning) is widespread on Hugging Face, creating models that outperform individual fine-tunes.
- **Model soups**: Averaging multiple training runs reduces variance and improves OOD robustness (Wortsman et al., 2022).
- **Evolutionary merging**: CMA-ES or genetic algorithms to search optimal merging coefficients per layer (Sakana AI's evolutionary model merge).
**Model merging has become a fundamental technique in the open-source AI ecosystem** — enabling the creation of capable multi-task models through simple weight arithmetic, democratizing model customization without the computational cost of multi-task training or the data requirements of comprehensive fine-tuning.