federated averaging
**FedAvg** (Federated Averaging) is the **foundational algorithm for federated learning** — each client performs multiple local SGD steps on their private data, then sends the updated model (or model delta) to the server, which averages the updates to produce the new global model.
**FedAvg Algorithm**
- **Server**: Send global model $w_t$ to a random subset of clients.
- **Client**: Each client $k$ runs $E$ epochs of SGD on their local data: $w_k = w_t - eta
abla L_k(w_t)$ (local training).
- **Communication**: Each client sends $w_k - w_t$ (model delta) to the server.
- **Aggregation**: Server averages: $w_{t+1} = w_t + frac{1}{K}sum_{k=1}^K (w_k - w_t)$ (weighted by dataset size).
**Why It Matters**
- **Communication Efficient**: Multiple local steps per communication round dramatically reduce communication.
- **Privacy**: Raw data never leaves the clients — only model updates are shared.
- **Heterogeneity Challenge**: Non-IID data across clients can cause FedAvg to diverge — motivating FedProx and SCAFFOLD.
**FedAvg** is **the workhorse of federated learning** — averaging locally trained models for collaborative learning without data sharing.