TIES-Merging (Trim, Elect Sign, and Merge) is a model merging method that resolves parameter conflicts when combining multiple task-specific models — addressing the interference problem where naively averaging conflicting parameter updates degrades performance.

How Does TIES-Merging Work?

• Trim: Remove (zero out) small-magnitude parameter changes that are likely noise.
• Elect Sign: For each parameter, determine the dominant sign (positive or negative) across all task vectors.
• Merge: Average only the parameters whose sign matches the elected dominant sign.
• Paper: Yadav et al. (2023).

Why It Matters

• Sign Conflict Resolution: When one task wants $+Delta$ and another wants $-Delta$, naive averaging gives $approx 0$ (destructive interference). TIES resolves this.
• Better Than Average: Significantly outperforms simple weight averaging and task arithmetic for multi-model merging.
• Scalable: Works with many task-specific models merged simultaneously.

TIES-Merging is conflict resolution for model merging — trimming noise, resolving sign conflicts, and averaging constructively for better multi-task models.