Federated Learning

What is Federated Learning? Training ML models across decentralized data sources without sharing raw data, preserving privacy while enabling collaborative learning.

How It Works

Central Server
    |
    | Model weights
    v
[Device 1] [Device 2] [Device 3]
    |           |           |
    | Local     | Local     | Local
    | training  | training  | training
    |           |           |
    v           v           v
Local updates aggregated by server

FedAvg Algorithm

def federated_averaging(server_model, clients, rounds=100):
    for round in range(rounds):
        client_weights = []

        # Each client trains locally
        for client in clients:
            local_model = copy(server_model)
            local_model.train(client.data)
            client_weights.append(local_model.state_dict())

        # Aggregate weights (simple average)
        averaged_weights = {}
        for key in server_model.state_dict():
            averaged_weights[key] = sum(
                w[key] for w in client_weights
            ) / len(clients)

        server_model.load_state_dict(averaged_weights)

    return server_model

Challenges

Privacy Enhancements

Differential Privacy in FL

def dp_sgd_update(gradients, clip_norm, noise_scale):
    # Clip gradient norm
    grad_norm = torch.norm(gradients)
    gradients = gradients * min(1, clip_norm / grad_norm)

    # Add noise
    noise = torch.randn_like(gradients) * noise_scale
    return gradients + noise

Frameworks

Use Cases

Best Practices

• Handle non-IID data with appropriate algorithms
• Compress updates for communication efficiency
• Add differential privacy for strong guarantees
• Validate federated models carefully