Model Merging

What is Model Merging? Combining multiple fine-tuned models into one without additional training.

Why Merge?

• Combine skills from different models
• Reduce deployment complexity
• Potentially improve generalization
• Cheap alternative to multi-task training

Merging Methods

Weight Averaging Simple average of model weights:

def average_merge(models):
    merged_state = {}
    n = len(models)

    for key in models[0].state_dict():
        weights = [m.state_dict()[key] for m in models]
        merged_state[key] = sum(weights) / n

    return merged_state

Task Arithmetic Add/subtract task-specific changes:

def task_arithmetic_merge(base, models, scaling_coefs):
    base_state = base.state_dict()

    merged_state = {k: v.clone() for k, v in base_state.items()}

    for model, coef in zip(models, scaling_coefs):
        task_vector = {}
        for key in model.state_dict():
            task_vector[key] = model.state_dict()[key] - base_state[key]
            merged_state[key] += coef * task_vector[key]

    return merged_state

TIES (Trim, Elect, Merge) More sophisticated merging:

def ties_merge(models, base, k=0.2):
    # 1. Trim: Keep only top-k% magnitude changes
    task_vectors = [trim_topk(m - base, k) for m in models]

    # 2. Elect: Resolve conflicts by sign voting
    elected = elect_signs(task_vectors)

    # 3. Merge: Average elected values
    merged_tv = average_matching(task_vectors, elected)

    return base + merged_tv

DARE (Drop And REscale) Random dropout of changes:

def dare_merge(models, base, drop_rate=0.9):
    task_vectors = [m - base for m in models]

    for tv in task_vectors:
        # Random dropout
        mask = torch.rand_like(tv) > drop_rate
        tv *= mask / (1 - drop_rate)  # Rescale

    return base + sum(task_vectors) / len(task_vectors)

Tools

mergekit Example

# merge.yaml
models:
  - model: base-model
    parameters:
      weight: 0.5
  - model: math-finetuned
    parameters:
      weight: 0.3
  - model: code-finetuned
    parameters:
      weight: 0.2

merge_method: linear
dtype: bfloat16

mergekit-yaml merge.yaml ./output_model

Best Practices

• Merge models from same base
• Experiment with different methods
• Evaluate on diverse benchmarks
• Consider task compatibility
• Try different weight coefficients