model merging
**Model Merging** is a **technique that combines multiple fine-tuned LLMs into a single model by interpolating or adding their weight spaces** — creating models with combined capabilities without any additional training.
**Why Model Merging?**
- Fine-tuning a base model for task A and task B separately produces specialized models.
- Naively combining capabilities requires multi-task fine-tuning (expensive, data needed).
- Model merging: Average the weights directly — surprisingly effective in the weight space.
**Merging Methods**
**Linear Merging (Model Soup)**:
- $\theta_{merged} = \frac{1}{n}\sum_i \theta_i$
- Simple average of fine-tuned model weights.
- Works well for models fine-tuned from the same base.
**SLERP (Spherical Linear Interpolation)**:
- $SLERP(\theta_A, \theta_B, t) = \frac{\sin((1-t)\Omega)}{\sin\Omega}\theta_A + \frac{\sin(t\Omega)}{\sin\Omega}\theta_B$
- Interpolates along the geodesic on a sphere — better for large weight differences.
**TIES-Merging**:
- Trims redundant parameters, resolves sign conflicts, then merges.
- Handles conflicting updates between multiple models more robustly.
**DARE**:
- Randomly drops and rescales delta weights before merging.
- Reduces parameter interference.
**Task Arithmetic**:
- Compute "task vectors": $\tau_A = \theta_{fine-tuned} - \theta_{base}$
- Add/subtract task vectors: $\theta_{merged} = \theta_{base} + \lambda_A \tau_A + \lambda_B \tau_B$
- Can "unlearn" a capability by subtracting its task vector.
**Practical Impact**
- WizardMath, WizardCoder, OpenHermes and many top open-source models use model merging.
- No training cost: Merge two 70B models in minutes on CPU.
- Competitive with multi-task fine-tuning in many settings.
Model merging is **a powerful, zero-cost technique for combining LLM capabilities** — it democratizes capability combination for practitioners without large compute budgets.