differential privacy
**Differential Privacy**
**What is Differential Privacy?**
A mathematical framework providing rigorous privacy guarantees, ensuring that the output of a computation is nearly the same whether or not any individual data point is included.
**Formal Definition**
A mechanism M is epsilon-differentially private if for all outputs S and datasets D, D_prime differing in one element:
```
P(M(D) in S) <= e^epsilon * P(M(D_prime) in S)
```
Lower epsilon = stronger privacy.
**Key Concepts**
| Concept | Description |
|---------|-------------|
| Epsilon (eps) | Privacy budget, lower is more private |
| Delta | Probability of failure |
| Sensitivity | Max change from one person |
| Noise | Added randomness for privacy |
**DP Mechanisms**
**Laplace Mechanism**
For numeric queries:
```python
def laplace_mechanism(true_value, sensitivity, epsilon):
scale = sensitivity / epsilon
noise = numpy.random.laplace(0, scale)
return true_value + noise
```
**Gaussian Mechanism**
For approximate DP:
```python
def gaussian_mechanism(true_value, sensitivity, epsilon, delta):
sigma = sensitivity * sqrt(2 * log(1.25 / delta)) / epsilon
noise = numpy.random.normal(0, sigma)
return true_value + noise
```
**DP-SGD (Differentially Private Training)**
```python
def dp_sgd_step(model, batch, clip_norm, noise_multiplier, lr):
# Compute per-sample gradients
per_sample_grads = compute_per_sample_gradients(model, batch)
# Clip each gradient
clipped_grads = [
g * min(1, clip_norm / g.norm())
for g in per_sample_grads
]
# Aggregate with noise
avg_grad = sum(clipped_grads) / len(batch)
noise = torch.randn_like(avg_grad) * clip_norm * noise_multiplier / len(batch)
noisy_grad = avg_grad + noise
# Update
for param, grad in zip(model.parameters(), noisy_grad):
param.data -= lr * grad
```
**Privacy Accounting**
Track cumulative privacy loss:
```python
from opacus.accountants import RDPAccountant
accountant = RDPAccountant()
for step in range(steps):
accountant.step(noise_multiplier, sample_rate)
epsilon, delta = accountant.get_privacy_spent(target_delta=1e-5)
print(f"Total privacy: eps={epsilon:.2f}, delta={delta}")
```
**Tools**
| Tool | Features |
|------|----------|
| Opacus | PyTorch DP training |
| TF Privacy | TensorFlow DP |
| PyDP | DP primitives |
| Tumult Analytics | DP analytics |
**Trade-offs**
| Higher Privacy | Lower Privacy |
|----------------|---------------|
| More noise | Less noise |
| Lower accuracy | Higher accuracy |
| Slower training | Faster training |
**Best Practices**
- Start with reasonable epsilon (1-10 for training)
- Use privacy accounting throughout
- Consider local vs central DP
- Validate utility on downstream tasks