accordion

**Accordion** is an **adaptive gradient compression framework that dynamically adjusts the compression ratio during training** — using more compression when the model is making rapid progress (gradient information is less critical) and less compression during delicate convergence phases. **How Accordion Works** - **Monitoring**: Track a training metric (gradient variance, loss change, learning rate) to assess the training phase. - **Adaptive Ratio**: High compression when gradients are informative (early training), low compression near convergence. - **Scheduler**: Compression ratio follows a schedule synchronized with the learning rate schedule. - **Any Compressor**: Works with any base compressor (top-K, random-K, PowerSGD, quantization). **Why It Matters** - **Optimal Efficiency**: Different training phases have different communication sensitivity — Accordion exploits this. - **No Accuracy Loss**: By being conservative when it matters and aggressive when it doesn't, Accordion achieves lossless training. - **Automatic**: No manual tuning of compression ratios — the framework adapts automatically. **Accordion** is **breathing with the training** — dynamically adjusting communication compression to match each training phase's sensitivity to gradient accuracy.