federated learning
**Federated Learning**
**What is Federated Learning?**
Training ML models across decentralized data sources without sharing raw data, preserving privacy while enabling collaborative learning.
**How It Works**
```
Central Server
|
| Model weights
v
[Device 1] [Device 2] [Device 3]
| | |
| Local | Local | Local
| training | training | training
| | |
v v v
Local updates aggregated by server
```
**FedAvg Algorithm**
```python
def federated_averaging(server_model, clients, rounds=100):
for round in range(rounds):
client_weights = []
# Each client trains locally
for client in clients:
local_model = copy(server_model)
local_model.train(client.data)
client_weights.append(local_model.state_dict())
# Aggregate weights (simple average)
averaged_weights = {}
for key in server_model.state_dict():
averaged_weights[key] = sum(
w[key] for w in client_weights
) / len(clients)
server_model.load_state_dict(averaged_weights)
return server_model
```
**Challenges**
| Challenge | Description |
|-----------|-------------|
| Non-IID data | Clients have different data distributions |
| System heterogeneity | Different compute/network capabilities |
| Communication cost | Sending model updates is expensive |
| Privacy attacks | Gradients can leak information |
**Privacy Enhancements**
| Technique | Protection |
|-----------|------------|
| Differential privacy | Add noise to updates |
| Secure aggregation | Encrypt updates |
| Local differential privacy | Noise at client |
| Compression | Reduce communication |
**Differential Privacy in FL**
```python
def dp_sgd_update(gradients, clip_norm, noise_scale):
# Clip gradient norm
grad_norm = torch.norm(gradients)
gradients = gradients * min(1, clip_norm / grad_norm)
# Add noise
noise = torch.randn_like(gradients) * noise_scale
return gradients + noise
```
**Frameworks**
| Framework | Features |
|-----------|----------|
| Flower | Flexible, framework-agnostic |
| PySyft | Privacy-focused |
| TensorFlow Federated | Google, production-ready |
| FATE | Enterprise FL |
**Use Cases**
| Domain | Application |
|--------|-------------|
| Healthcare | Train on hospital data without sharing |
| Mobile | Keyboard prediction with user data |
| Finance | Fraud detection across institutions |
| IoT | Edge device collaborative learning |
**Best Practices**
- Handle non-IID data with appropriate algorithms
- Compress updates for communication efficiency
- Add differential privacy for strong guarantees
- Validate federated models carefully