federated learning
**Federated Learning** is the **distributed machine learning paradigm where models are trained across many decentralized devices (phones, hospitals, banks) without raw data ever leaving the local device** — enabling collaborative model improvement while preserving data privacy, regulatory compliance (GDPR/HIPAA), and data sovereignty, with the central server only receiving model updates rather than sensitive user data.
**How Federated Learning Works (FedAvg)**
1. **Server distributes** current global model weights to selected client devices.
2. **Clients train locally** on their private data for E epochs (typically 1-5).
3. **Clients send model updates** (weight deltas or gradients) back to server.
4. **Server aggregates** updates: $w_{global}^{t+1} = \sum_{k=1}^{K} \frac{n_k}{n} w_k^{t+1}$.
- Weighted average by number of local samples per client.
5. Repeat for multiple communication rounds until convergence.
**Key Challenges**
| Challenge | Description | Mitigation |
|-----------|------------|------------|
| Non-IID data | Clients have different data distributions | FedProx, SCAFFOLD, personalization |
| Communication cost | Model updates are large, networks are slow | Gradient compression, quantization |
| Stragglers | Some devices are slower than others | Async aggregation, client sampling |
| Privacy leakage | Gradients can reveal information about data | Differential privacy, secure aggregation |
| Heterogeneous devices | Different compute/memory capabilities | Adaptive model sizes, knowledge distillation |
**Non-IID Problem (The Core Challenge)**
- IID (Independent and Identically Distributed): Each client has representative sample of global data.
- Non-IID (reality): User A has mostly cat photos, User B has mostly food photos.
- Non-IID causes: Client models diverge → averaging produces poor global model.
- Solutions: FedProx (proximity regularization), SCAFFOLD (variance reduction), local fine-tuning.
**Privacy Enhancements**
- **Secure Aggregation**: Cryptographic protocol ensures server sees only the aggregate update, not individual client updates.
- **Differential Privacy**: Add calibrated noise to client updates → formal privacy guarantee (ε-DP).
- Trade-off: More privacy (smaller ε) → more noise → lower model accuracy.
- **Trusted Execution Environments**: Run aggregation in secure enclaves (SGX, TrustZone).
**Real-World Deployments**
- **Google Gboard**: Next-word prediction trained on-device via federated learning.
- **Apple**: Siri improvement, QuickType suggestions — federated with differential privacy.
- **Healthcare**: Hospital networks training diagnostic models without sharing patient data.
- **Financial**: Banks collaboratively detecting fraud without sharing transaction records.
Federated learning is **the enabling technology for privacy-preserving AI at scale** — as data privacy regulations tighten globally and data remains the most sensitive asset organizations hold, federated learning provides the only viable path for collaborative model training without centralized data collection.