merging
**Model Merging**
**What is Model Merging?**
Combining multiple fine-tuned models into one without additional training.
**Why Merge?**
- Combine skills from different models
- Reduce deployment complexity
- Potentially improve generalization
- Cheap alternative to multi-task training
**Merging Methods**
**Weight Averaging**
Simple average of model weights:
```python
def average_merge(models):
merged_state = {}
n = len(models)
for key in models[0].state_dict():
weights = [m.state_dict()[key] for m in models]
merged_state[key] = sum(weights) / n
return merged_state
```
**Task Arithmetic**
Add/subtract task-specific changes:
```python
def task_arithmetic_merge(base, models, scaling_coefs):
base_state = base.state_dict()
merged_state = {k: v.clone() for k, v in base_state.items()}
for model, coef in zip(models, scaling_coefs):
task_vector = {}
for key in model.state_dict():
task_vector[key] = model.state_dict()[key] - base_state[key]
merged_state[key] += coef * task_vector[key]
return merged_state
```
**TIES (Trim, Elect, Merge)**
More sophisticated merging:
```python
def ties_merge(models, base, k=0.2):
# 1. Trim: Keep only top-k% magnitude changes
task_vectors = [trim_topk(m - base, k) for m in models]
# 2. Elect: Resolve conflicts by sign voting
elected = elect_signs(task_vectors)
# 3. Merge: Average elected values
merged_tv = average_matching(task_vectors, elected)
return base + merged_tv
```
**DARE (Drop And REscale)**
Random dropout of changes:
```python
def dare_merge(models, base, drop_rate=0.9):
task_vectors = [m - base for m in models]
for tv in task_vectors:
# Random dropout
mask = torch.rand_like(tv) > drop_rate
tv *= mask / (1 - drop_rate) # Rescale
return base + sum(task_vectors) / len(task_vectors)
```
**Tools**
| Tool | Features |
|------|----------|
| mergekit | CLI for model merging |
| Model Stock | Pre-computed merges |
| PEFT merge | Merge LoRA adapters |
**mergekit Example**
```yaml
# merge.yaml
models:
- model: base-model
parameters:
weight: 0.5
- model: math-finetuned
parameters:
weight: 0.3
- model: code-finetuned
parameters:
weight: 0.2
merge_method: linear
dtype: bfloat16
```
```bash
mergekit-yaml merge.yaml ./output_model
```
**Best Practices**
- Merge models from same base
- Experiment with different methods
- Evaluate on diverse benchmarks
- Consider task compatibility
- Try different weight coefficients