federated learning basics

**Federated Learning** — a distributed training approach where models are trained across many decentralized devices (phones, hospitals, banks) without sharing raw data, preserving privacy. **How It Works** 1. Server sends global model to N client devices 2. Each device trains on its local data for a few epochs 3. Devices send only model updates (gradients/weights) back to server — NOT the raw data 4. Server aggregates updates (FedAvg: weighted average) → new global model 5. Repeat for many rounds **Why Federated Learning?** - **Privacy**: Raw data never leaves the device (medical records, financial data, personal messages) - **Regulation**: GDPR, HIPAA compliance — data can't be centralized - **Scale**: Billions of mobile devices as training nodes (Google Keyboard predictions trained this way) **Challenges** - **Non-IID data**: Each device has different data distribution (heterogeneous) - **Communication cost**: Sending model updates is expensive over mobile networks - **Stragglers**: Some devices are slow or drop out - **Privacy attacks**: Gradient inversion can partially reconstruct training data **Real Applications** - Google Gboard: Next-word prediction trained on-device - Apple: Siri improvements without collecting voice data - Healthcare: Multi-hospital medical imaging models **Federated learning** makes it possible to train AI on sensitive data that could never be collected into a single dataset.