Model Merging is a technique that combines multiple fine-tuned LLMs into a single model by interpolating or adding their weight spaces — creating models with combined capabilities without any additional training.

Why Model Merging?

• Fine-tuning a base model for task A and task B separately produces specialized models.
• Naively combining capabilities requires multi-task fine-tuning (expensive, data needed).
• Model merging: Average the weights directly — surprisingly effective in the weight space.

Merging Methods

Linear Merging (Model Soup):

• $\theta_{merged} = \frac{1}{n}\sum_i \theta_i$
• Simple average of fine-tuned model weights.
• Works well for models fine-tuned from the same base.

SLERP (Spherical Linear Interpolation):

• $SLERP(\theta_A, \theta_B, t) = \frac{\sin((1-t)\Omega)}{\sin\Omega}\theta_A + \frac{\sin(t\Omega)}{\sin\Omega}\theta_B$
• Interpolates along the geodesic on a sphere — better for large weight differences.

TIES-Merging:

• Trims redundant parameters, resolves sign conflicts, then merges.
• Handles conflicting updates between multiple models more robustly.

DARE:

• Randomly drops and rescales delta weights before merging.
• Reduces parameter interference.

Task Arithmetic:

• Compute "task vectors": $\tau_A = \theta_{fine-tuned} - \theta_{base}$
• Add/subtract task vectors: $\theta_{merged} = \theta_{base} + \lambda_A \tau_A + \lambda_B \tau_B$
• Can "unlearn" a capability by subtracting its task vector.

Practical Impact

• WizardMath, WizardCoder, OpenHermes and many top open-source models use model merging.
• No training cost: Merge two 70B models in minutes on CPU.
• Competitive with multi-task fine-tuning in many settings.

Model merging is a powerful, zero-cost technique for combining LLM capabilities — it democratizes capability combination for practitioners without large compute budgets.