federated learning privacy

**Federated Learning (FL)** is the **distributed machine learning paradigm where models are trained across multiple decentralized devices or institutions without centralizing the raw data — each participant trains locally on their private data and shares only model updates (gradients or weights) with a central server that aggregates them, preserving data privacy while enabling collaborative model improvement across organizational and regulatory boundaries**. **Why Federated Learning Exists** Traditional ML requires centralizing all training data in one location. This is impossible when: - **Regulatory constraints**: GDPR, HIPAA, or CCPA prohibit data sharing across jurisdictions or organizations. - **Privacy sensitivity**: Medical records, financial transactions, and personal communications cannot leave the source device/institution. - **Data volume**: Mobile devices collectively hold petabytes of data that is impractical to centralize. - **Competitive concerns**: Multiple hospitals want to collaboratively train a better diagnostic model without sharing their patients' data with competitors. **Federated Averaging (FedAvg)** The foundational FL algorithm: 1. Server sends the current global model to a random subset of clients. 2. Each client trains the model on its local data for E epochs (local SGD). 3. Clients send their updated model weights (or weight deltas) back to the server. 4. Server averages the client updates: w_global = (1/K) Σ wₖ, weighted by each client's dataset size. 5. Repeat until convergence. **Challenges and Solutions** - **Non-IID Data**: Client datasets have different distributions (a hospital specializing in cardiac cases vs. oncology). FedAvg can diverge. Solutions: FedProx (proximal regularization), SCAFFOLD (variance reduction), personalized federated learning (per-client adaptation layers). - **Communication Efficiency**: Sending full model updates (hundreds of MB for large models) is expensive over mobile networks. Solutions: gradient compression (top-K sparsification, quantization), federated distillation (share logits instead of weights), increasing local computation (E>1) to reduce round trips. - **Client Heterogeneity**: Devices have different compute capabilities and availability. Asynchronous FL allows clients to contribute updates at their own pace; knowledge distillation enables different model architectures per client. - **Privacy Attacks**: Even without raw data, model gradients can leak information (gradient inversion attacks can reconstruct training images). Defenses: - **Differential Privacy**: Add calibrated noise to gradient updates, providing mathematical privacy guarantees (ε-differential privacy). - **Secure Aggregation**: Cryptographic protocols ensure the server can compute the aggregate without seeing individual client updates. - **Trusted Execution Environments**: Hardware enclaves (Intel SGX) process aggregation in isolated, verifiable environments. **Production Deployments** - **Google Gboard**: Next-word prediction trained across millions of Android devices using federated learning. The model improves from global keyboard usage without Google seeing what users type. - **Apple**: On-device ML models for Siri, QuickType, and photo features trained using privacy-preserving federated approaches. Federated Learning is **the privacy-preserving training paradigm that resolves the fundamental tension between data-hungry ML and data-protective regulation** — enabling models to learn from the world's distributed data without that data ever leaving its source.