ties-merging

**TIES-Merging** (Trim, Elect Sign, and Merge) is a **model merging method that resolves parameter conflicts when combining multiple task-specific models** — addressing the interference problem where naively averaging conflicting parameter updates degrades performance. **How Does TIES-Merging Work?** - **Trim**: Remove (zero out) small-magnitude parameter changes that are likely noise. - **Elect Sign**: For each parameter, determine the dominant sign (positive or negative) across all task vectors. - **Merge**: Average only the parameters whose sign matches the elected dominant sign. - **Paper**: Yadav et al. (2023). **Why It Matters** - **Sign Conflict Resolution**: When one task wants $+Delta$ and another wants $-Delta$, naive averaging gives $approx 0$ (destructive interference). TIES resolves this. - **Better Than Average**: Significantly outperforms simple weight averaging and task arithmetic for multi-model merging. - **Scalable**: Works with many task-specific models merged simultaneously. **TIES-Merging** is **conflict resolution for model merging** — trimming noise, resolving sign conflicts, and averaging constructively for better multi-task models.