pad token

**PAD token** is the **special token used to pad variable-length sequences to uniform batch shapes for efficient parallel processing** - it is fundamental for batching in training and inference. **What Is PAD token?** - **Definition**: Reserved token inserted where no real content exists to align sequence lengths. - **Batching Role**: Enables vectorized computation by forming fixed-size tensors. - **Masking Requirement**: Attention masks ensure PAD positions do not affect model predictions. - **Placement Strategy**: Padding can be left or right aligned depending on model and runtime. **Why PAD token Matters** - **Compute Efficiency**: Uniform shapes improve accelerator utilization and throughput. - **Pipeline Simplicity**: Batch operations are easier when sequence dimensions are standardized. - **Correctness**: Proper masking prevents padding artifacts from leaking into outputs. - **Serving Scalability**: Dynamic batching relies on safe and predictable padding behavior. - **Compatibility**: PAD token IDs must align across tokenizer, model config, and runtime. **How It Is Used in Practice** - **Mask Validation**: Test that padded positions are fully ignored in attention and loss computation. - **Alignment Tuning**: Choose left or right padding based on cache and decode characteristics. - **Runtime Checks**: Audit PAD usage in batch constructors to prevent silent shape bugs. PAD token is **a core batching primitive in sequence-model infrastructure** - correct PAD handling is essential for both performance and output integrity.