privacy budget
**Privacy Budget** is the **quantitative measure that tracks the cumulative privacy loss of a differential privacy system** — expressed as the epsilon (ε) parameter that bounds how much information about any individual can leak through the system's outputs, where each query, training step, or data access consumes a portion of the finite budget, and once exhausted, no further computations can be performed without violating privacy guarantees.
**What Is a Privacy Budget?**
- **Definition**: The total amount of privacy loss (ε) that a system is allowed to incur across all operations on a private dataset.
- **Core Concept**: Every interaction with private data leaks some information — the privacy budget sets a hard limit on total leakage.
- **Key Parameter**: Epsilon (ε) — lower values mean stronger privacy (ε=0.1 is very strong, ε=10 is weak).
- **Finite Resource**: Unlike computational budgets that can be replenished, privacy budget is a one-way ratchet — once spent, protection is permanently reduced.
**Why Privacy Budget Matters**
- **Accountability**: Provides a concrete, measurable limit on how much privacy can be lost.
- **Resource Management**: Forces organizations to prioritize which analyses and models are worth the privacy cost.
- **Regulatory Compliance**: Enables demonstrable compliance with privacy regulations through quantifiable guarantees.
- **Composition Control**: Without budget tracking, repeated queries could cumulatively destroy privacy.
- **Trust Building**: Users can be assured their data is protected up to a specified, auditable level.
**How Privacy Budget Works**
| Concept | Explanation | Analogy |
|---------|-------------|---------|
| **Total Budget (ε_total)** | Maximum allowed cumulative privacy loss | Total money in a bank account |
| **Per-Query Cost** | Privacy loss from each operation | Each purchase deducts from balance |
| **Remaining Budget** | ε_total minus cumulative spending | Current account balance |
| **Budget Exhaustion** | No more queries allowed | Account is empty |
| **Composition** | How individual costs accumulate | How purchases add up |
**Composition Theorems**
- **Basic Composition**: For k queries each with privacy ε_i, total privacy is Σε_i (linear — pessimistic).
- **Advanced Composition**: For k queries each with privacy ε, total is O(ε√(k·log(1/δ))) (sublinear — tighter).
- **Rényi Composition**: Uses Rényi divergence for even tighter privacy accounting.
- **Moments Accountant**: Numerical tracking providing the tightest known composition bounds for DP-SGD.
**Budget Allocation Strategies**
- **Equal Allocation**: Divide budget equally across anticipated queries.
- **Priority-Based**: Allocate more budget to high-value analyses, less to exploratory queries.
- **Adaptive**: Dynamically allocate budget based on query importance and remaining balance.
- **Hierarchical**: Set organizational budget, then sub-allocate to teams and projects.
**Practical Considerations**
- **Setting ε**: No universal "right" value — depends on data sensitivity, threat model, and utility requirements.
- **Apple**: Uses ε=2-8 for local differential privacy in iOS analytics.
- **Google**: Uses ε=0.5-8 for RAPPOR and Chrome data collection.
- **US Census**: Used ε≈19.61 for 2020 Census disclosure avoidance.
Privacy Budget is **the fundamental resource that makes differential privacy practical** — providing the accounting framework that transforms abstract privacy guarantees into concrete, manageable limits that organizations can allocate, track, and audit across all operations on sensitive data.