Differential Privacy Privacy-Preserving Machine Learning

# Differential Privacy: Privacy-Preserving Machine Learning

## Introduction & Motivation

Differential privacy adds noise to gradients/data; formal privacy guarantee. DP-SGD: clip gradients, add Gaussian noise per batch. Quantifies privacy-utility trade-off. Applications: federated learning, sensitive databases, medical data.

Motivation: Models memorize training data; privacy attack risk. DP provides formal guarantee independent of adversary's auxiliary knowledge.

Applications: Healthcare analytics, financial systems, census data, federated learning.

---

## Core Concepts & Theory

### Differential Privacy Definition

Algorithm is (ε,δ)-DP if output distributions on adjacent datasets differ by ≤ exp(ε), except δ-probability event.

### DP-SGD

Clip gradients per sample (L2 norm); add Gaussian noise; privacy budget ε accumulates.

### Privacy Amplification

Subsampling batch during SGD amplifies privacy; composition reduces ε.

---

## Mathematical Formulation

(ε,δ)-differential privacy:
$$\Pr[\mathcal{A}(D) \in S] \leq e^\epsilon \Pr[\mathcal{A}(D') \in S] + \delta$$

DP-SGD update (clipped + noised):
$$ heta_t = heta_{t-1} - \eta \left( \frac{1}{B} \sum_{i \in ext{batch}} ext{clip}\left( abla_i, C ight) + \mathcal{N}(0, \sigma^2 C^2 I) ight)$$

Privacy budget (moments accountant):
$$\epsilon(T) = \sqrt{T} ext{erf}^{-1}(2\delta - 1) \cdot \frac{q \sigma}{\sqrt{2}}$$

---

## Advanced Theory & Extensions

### Renyi Differential Privacy

Tighter analysis; composition rules.

### Local Differential Privacy

Noise added locally; server never sees raw data.

### Group Privacy

Privacy-utility trade-off for groups vs. individuals.

---

## Computational Considerations

Clipping: O(d) per sample (compute norm).

Noise addition: O(d) per batch.

Privacy accounting: O(T) moments computation.

---

## Practical Implementation Strategies

### Hyperparameter Selection

C (clipping norm), σ (noise scale) jointly determine ε-δ.

### Privacy Budget Allocation

Distribute ε across epochs; higher early, lower later.

### Batch Effects

Larger batches → more ε consumption per update.

---

## Benchmark Datasets & Evaluation

MNIST DP-SGD: Accuracy-privacy trade-off; ε=1 achieves ~95% with clipping.

CIFAR-10: Accuracy degrades ~5-10% at ε=1.

Metrics: Final accuracy at (ε,δ), convergence rate, privacy cost.

---

## Key Challenges & Limitations

### Accuracy-Privacy Trade-off

DP incurs accuracy loss; ε determines magnitude.

### Hyperparameter Tuning

C, σ interdependent; sensitive to dataset.

### Composition Overhead

Multiple epochs → rapid ε depletion; limits training.

---

## Hyperparameter Tuning

Clipping norm C: 0.1-1.0; dataset-dependent.

Noise multiplier σ: 0.5-2.0; higher → stronger privacy.

Target (ε,δ): ε ∈ [0.1, 10], δ ∈ [1e-5, 1e-3].

---

## Real-World Applications & Case Studies

Census 2020: U.S. Census used DP for population counts.

Healthcare Analytics: Sensitive patient data; DP ensures privacy.

Federated Learning: DP-SGD on device updates; central privacy guarantee.

---

## Integration with Other Methods

DP + Federated → privacy-preserving distributed training.

DP + Differential Privacy Amplification → composition across rounds.

---

## Summary & Key Takeaways

Differential privacy adds formal privacy guarantees via gradient clipping and noise injection, with DP-SGD providing composable privacy budget tracking across training rounds.

Principles:
1. (ε,δ)-DP: formal privacy guarantee; ε quantifies distinguishability.
2. DP-SGD: clip per-sample gradients, add Gaussian noise.
3. Privacy budget accumulates; composition analysis essential.
4. Accuracy-privacy trade-off; practical ε ∈ [0.1, 10].
5. Clipping norm, noise scale jointly control privacy-utility.

---

---

## Appendix: Practical Labs

### Lab 1: Gradient Clipping

import torch
import numpy as np

def clip_gradient(grad, max_norm=1.0):
 """Clip gradient to max L2 norm"""
 grad_norm = torch.norm(grad)
 scale = min(1.0, max_norm / (grad_norm + 1e-8))
 return scale * grad

# Test
grad = torch.randn(100)
clipped = clip_gradient(grad, max_norm=1.0)

print(f"Original norm: {torch.norm(grad):.4f}, Clipped: {torch.norm(clipped):.4f}")
assert torch.norm(clipped) <= 1.0 + 1e-5, "Should respect max norm"
print("✓ Gradient clipping working")

if __name__ == "__main__":
 print("Lab 1: Clipping - PASSED")

### Lab 2: Noise Addition

import torch
import numpy as np

def add_dp_noise(grad, clipping_norm=1.0, noise_multiplier=1.0):
 """Add Gaussian noise for DP"""
 grad_norm = torch.norm(grad)
 scale = min(1.0, clipping_norm / (grad_norm + 1e-8))
 clipped = scale * grad
 
 sigma = noise_multiplier * clipping_norm
 noise = torch.randn_like(clipped) * sigma
 
 return clipped + noise

# Test
grad = torch.randn(100)
noised = add_dp_noise(grad, clipping_norm=1.0, noise_multiplier=0.5)

print(f"Noised gradient norm: {torch.norm(noised):.4f}")
assert torch.isfinite(noised).all(), "Should be finite"
print("✓ Noise addition working")

if __name__ == "__main__":
 print("Lab 2: Noise - PASSED")

### Lab 3: Privacy Accounting

import numpy as np
from scipy.special import erf

def privacy_budget_moments(n_steps, sampling_rate, noise_multiplier, delta=1e-5):
 """Estimate ε via moments accountant"""
 # Simplified: sqrt(2 * ln(1/delta)) * q * sigma / sqrt(T)
 epsilon = np.sqrt(2 * np.log(1/delta)) * sampling_rate * noise_multiplier / np.sqrt(n_steps)
 return epsilon

# Test
eps = privacy_budget_moments(n_steps=1000, sampling_rate=0.01, noise_multiplier=1.0, delta=1e-5)

print(f"Privacy budget ε: {eps:.4f}")
assert eps > 0, "Should be positive"
assert np.isfinite(eps), "Should be finite"
print("✓ Privacy accounting working")

if __name__ == "__main__":
 print("Lab 3: Privacy Accounting - PASSED")

### Lab 4: Accuracy-Privacy Trade-off

import numpy as np

def accuracy_privacy_tradeoff(noise_multipliers):
 """Simulate accuracy vs privacy (dummy model)"""
 
 # Simplified model: accuracy degrades with DP noise
 baseline_acc = 0.95
 
 results = []
 for sigma in noise_multipliers:
 # Mock: accuracy degrades linearly with noise
 accuracy = baseline_acc * np.exp(-0.5 * sigma)
 epsilon = 1.0 / (sigma + 1e-8)
 
 results.append({'sigma': sigma, 'accuracy': accuracy, 'epsilon': epsilon})
 
 return results

# Test
trade_offs = accuracy_privacy_tradeoff([0.5, 1.0, 1.5, 2.0])

print(f"Trade-off at σ=1.0: accuracy {trade_offs[1]['accuracy']:.3f}, ε={trade_offs[1]['epsilon']:.3f}")
assert len(trade_offs) == 4, "Should have 4 points"
assert all(0 < r['accuracy'] < 1 for r in trade_offs), "Accuracy should be valid"
print("✓ Accuracy-privacy trade-off working")

if __name__ == "__main__":
 print("Lab 4: Trade-off - PASSED")