conditional position encoding

**Conditional Position Encoding (CPE)** is a **position encoding method that generates position-dependent features conditioned on the input content** — using a depth-wise convolution on the input tokens to implicitly encode positional information. **How Does CPE Work?** - **Mechanism**: Apply a depth-wise convolution (e.g., 3×3) to the token sequence/grid. - **Implicit Position**: The convolution with zero-padding naturally encodes boundary positions. Interior positions are derived from local neighborhood structure. - **Dynamic**: The positional information adapts to the input content, unlike fixed sinusoidal encodings. - **Paper**: Chu et al. (2021, Conditional Positional Encodings for ViT). **Why It Matters** - **No Explicit PE**: Eliminates the need for explicit position embeddings (learnable or sinusoidal). - **Resolution Flexible**: Works at any resolution without interpolation (CNNs are inherently resolution-agnostic). - **Integration**: Can be added at any layer, not just the input, providing positional information throughout the network. **CPE** is **implicit position from local structure** — using convolutions to derive positional information from the spatial arrangement of features.