gradient compression for privacy

**Gradient Compression for Privacy** is the **use of gradient compression techniques (sparsification, quantization) to reduce privacy leakage in distributed training** — by transmitting only partial gradient information, less private data can be reconstructed from the shared updates. **Compression as Privacy Mechanism** - **Top-K Sparsification**: Send only the K largest gradient components — attackers cannot reconstruct full gradient. - **Random Sparsification**: Randomly sample gradient components to share — adds uncertainty for attackers. - **Quantization**: Reduce gradient precision (e.g., 1-bit SGD) — less information per component. - **Combined**: Use compression with DP noise for amplified privacy (privacy amplification by subsampling). **Why It Matters** - **Dual Benefit**: Gradient compression reduces both communication cost AND privacy leakage. - **Gradient Inversion**: Full-precision gradients can be inverted to reconstruct training data — compression makes inversion harder. - **Practical**: Compression is already used for efficiency in distributed training — the privacy benefit comes for free. **Gradient Compression for Privacy** is **leaking less by sending less** — using gradient compression to simultaneously improve communication efficiency and data privacy.