federated learning privacy

**Federated Learning** is **the distributed machine learning paradigm where multiple clients (devices or organizations) collaboratively train a shared model without exchanging raw data — each client trains on local data and shares only model updates (gradients or weights) with a central server, preserving data privacy while leveraging the collective knowledge of all participants**. **Federated Averaging (FedAvg):** - **Algorithm**: server distributes global model to selected clients → each client performs E epochs of local SGD on its private data → clients send model updates to server → server averages updates weighted by local dataset size → repeat - **Communication Rounds**: typical convergence requires 100-1000 communication rounds — each round involves model distribution (server→clients) and update collection (clients→server); communication of full model weights dominates system cost - **Client Selection**: each round samples a fraction C of clients (typically 1-10%) — random selection provides unbiased gradient estimates; clients with more data may be preferentially selected for faster convergence - **Local Epochs**: more local epochs (E>1) reduce communication rounds but increase divergence between client models — client drift accumulates when local data distributions differ significantly from the global distribution **Data Heterogeneity Challenges:** - **Non-IID Data**: client data distributions are typically non-identical — some clients may have only certain classes or heavily skewed distributions; non-IID data causes client model divergence and slower convergence - **Label Skew**: different clients have different label distributions — solutions: sharing a small global dataset, FedProx with proximal term preventing excessive divergence from global model, SCAFFOLD using control variates for variance reduction - **Feature Skew**: same labels but different feature distributions across clients — different lighting conditions, camera angles, or demographics; domain adaptation techniques help bridge feature gaps - **Quantity Skew**: vastly different dataset sizes across clients — small-data clients may overfit locally; weighted averaging mitigates by giving less weight to small datasets **Privacy and Security:** - **Privacy Guarantees**: raw data never leaves the client — but model updates can leak information; gradient inversion attacks can reconstruct training images from shared gradients - **Differential Privacy**: add calibrated noise to model updates before sharing — provides mathematical privacy guarantee (ε-differential privacy); traded against model accuracy; typical ε=1-10 for practical use - **Secure Aggregation**: cryptographic protocol ensures server only sees the aggregate of all client updates, not individual contributions — protects against honest-but-curious server; adds 2-5× communication overhead - **Byzantine Resilience**: robust aggregation methods (trimmed mean, Krum, median) tolerate malicious clients submitting poisoned updates — critical for open participation scenarios **Federated learning enables AI model training in privacy-sensitive domains (healthcare, finance, mobile) where data cannot be centralized — organizations like Google (Gboard), Apple (Siri), and hospitals collaborating on medical AI already deploy federated learning in production systems.**