federated learning
**Federated Learning** is the **distributed machine learning paradigm where models are trained across multiple decentralized devices or data silos without transferring raw data to a central server**, preserving data privacy by communicating only model updates (gradients or weights) — enabling collaborative learning across hospitals, mobile devices, financial institutions, and other privacy-sensitive domains.
**The FedAvg Algorithm** (foundational federated learning):
1. **Server distributes** current global model weights to selected client devices
2. **Each client trains** the model locally on its private data for E local epochs with learning rate η
3. **Clients send** updated model weights (or weight deltas) back to the server
4. **Server aggregates** client updates: w_global = Σ(n_k/n) · w_k (weighted average by client data size)
5. Repeat for T communication rounds
**Communication Efficiency**: Communication is the primary bottleneck — clients may be on slow mobile networks. Mitigation strategies: **local SGD** (more local epochs before communication — trades freshness for less communication); **gradient compression** (quantization, sparsification — 10-100× communication reduction); **partial model updates** (clients train and send only a subset of parameters); and **one-shot federated learning** (clients train independently, aggregate once).
**Non-IID Data Challenge**: The most fundamental difficulty. Federated data is rarely independently and identically distributed: hospital A may see mostly cardiac cases while hospital B sees neurological cases; mobile users have different typing patterns, languages, and usage frequency. Non-IID data causes **client drift** — local models overfit to local distributions and diverge from each other, degrading aggregated model quality.
**Non-IID Mitigations**:
| Method | Approach | Overhead |
|--------|---------|----------|
| **FedProx** | Add proximal term to keep local models near global | Minimal |
| **SCAFFOLD** | Variance reduction via control variates | 2× communication |
| **FedBN** | Keep batch norm local, share other layers | None |
| **Personalized FL** | Learn personalized models per client | Storage |
| **FedMA** | Match and average neurons by alignment | Computation |
**Privacy Guarantees**: FedAvg alone is not sufficient for formal privacy — model updates can leak information about training data (gradient inversion attacks can reconstruct training images from shared gradients). Stronger privacy requires: **Differential Privacy** (add calibrated noise to gradients — provides mathematical privacy guarantee at accuracy cost); **Secure Aggregation** (cryptographic protocol ensuring server sees only the aggregate, not individual updates); and **Trusted Execution Environments** (hardware enclaves for secure computation).
**Cross-Device vs. Cross-Silo**:
| Dimension | Cross-Device | Cross-Silo |
|-----------|-------------|------------|
| Clients | Millions (phones) | 2-100 (organizations) |
| Availability | Intermittent | Always on |
| Data per client | Small (KB-MB) | Large (GB-TB) |
| Compute | Limited | High |
| Example | Google Keyboard | Multi-hospital research |
**Federated learning enables collaboration without data centralization — transforming the economics of AI training for domains where data sharing is legally prohibited, ethically questionable, or commercially sensitive, while demonstrating that privacy and model quality need not be mutually exclusive.**