Differential Privacy

What is Differential Privacy? A mathematical framework providing rigorous privacy guarantees, ensuring that the output of a computation is nearly the same whether or not any individual data point is included.

Formal Definition A mechanism M is epsilon-differentially private if for all outputs S and datasets D, D_prime differing in one element:

P(M(D) in S) <= e^epsilon * P(M(D_prime) in S)

Lower epsilon = stronger privacy.

Key Concepts

DP Mechanisms

Laplace Mechanism For numeric queries:

def laplace_mechanism(true_value, sensitivity, epsilon):
    scale = sensitivity / epsilon
    noise = numpy.random.laplace(0, scale)
    return true_value + noise

Gaussian Mechanism For approximate DP:

def gaussian_mechanism(true_value, sensitivity, epsilon, delta):
    sigma = sensitivity * sqrt(2 * log(1.25 / delta)) / epsilon
    noise = numpy.random.normal(0, sigma)
    return true_value + noise

DP-SGD (Differentially Private Training)

def dp_sgd_step(model, batch, clip_norm, noise_multiplier, lr):
    # Compute per-sample gradients
    per_sample_grads = compute_per_sample_gradients(model, batch)

    # Clip each gradient
    clipped_grads = [
        g * min(1, clip_norm / g.norm())
        for g in per_sample_grads
    ]

    # Aggregate with noise
    avg_grad = sum(clipped_grads) / len(batch)
    noise = torch.randn_like(avg_grad) * clip_norm * noise_multiplier / len(batch)

    noisy_grad = avg_grad + noise

    # Update
    for param, grad in zip(model.parameters(), noisy_grad):
        param.data -= lr * grad

Privacy Accounting Track cumulative privacy loss:

from opacus.accountants import RDPAccountant

accountant = RDPAccountant()
for step in range(steps):
    accountant.step(noise_multiplier, sample_rate)

epsilon, delta = accountant.get_privacy_spent(target_delta=1e-5)
print(f"Total privacy: eps={epsilon:.2f}, delta={delta}")

Tools

Trade-offs

Best Practices

• Start with reasonable epsilon (1-10 for training)
• Use privacy accounting throughout
• Consider local vs central DP
• Validate utility on downstream tasks