per-tensor quantization
**Per-tensor quantization** uses a **single set of quantization parameters** (scale and zero-point) for an entire tensor, regardless of its shape or the variance across its dimensions. This is the simplest and most common quantization granularity.
**How It Works**
For a tensor $T$ with arbitrary shape:
$$q = ext{round}(T / s + z)$$
Where:
- $s$ is the **scale factor** (computed from the tensor's min/max values).
- $z$ is the **zero-point offset** (for asymmetric quantization).
**Scale Calculation**
For 8-bit quantization:
$$s = frac{max(T) - min(T)}{255}$$
(For symmetric quantization, use $max(|T|)$ instead.)
**Advantages**
- **Simplicity**: One scale and zero-point for the entire tensor — minimal storage overhead.
- **Fast Inference**: Dequantization is straightforward with no per-channel or per-element overhead.
- **Hardware Friendly**: Most quantization-aware hardware accelerators (TPUs, NPUs) are optimized for per-tensor quantization.
**Disadvantages**
- **Suboptimal for Heterogeneous Data**: If different regions of the tensor have very different value ranges, per-tensor quantization wastes precision. For example, if one channel has values in [-0.1, 0.1] and another in [-10, 10], the shared scale must accommodate [-10, 10], losing precision for the first channel.
- **Outliers**: A single outlier value can dominate the scale calculation, reducing precision for the majority of values.
**When to Use Per-Tensor**
- **Activations**: Standard choice for activation quantization because per-channel activations would require runtime overhead.
- **Small Tensors**: For tensors with relatively uniform value distributions.
- **Hardware Constraints**: When deploying to hardware that only supports per-tensor quantization.
**Comparison to Per-Channel**
| Aspect | Per-Tensor | Per-Channel |
|--------|------------|-------------|
| Parameters | 1 scale + 1 zero-point | N scales + N zero-points (N = channels) |
| Accuracy | Lower (for heterogeneous data) | Higher |
| Speed | Fastest | Slightly slower |
| Storage | Minimal | Small overhead |
| Use Case | Activations, uniform data | Weights, heterogeneous data |
**Example**
For a weight tensor with shape [64, 128, 3, 3] (64 output channels):
- **Per-Tensor**: Compute $min$ and $max$ across all 73,728 values, derive one scale.
- **Per-Channel**: Compute $min$ and $max$ for each of the 64 output channels separately, derive 64 scales.
Per-tensor quantization is the **default choice for activations** and a reasonable baseline for weights, though per-channel quantization typically provides better accuracy for weights.