bos / eos tokens

BOS (beginning-of-sequence) and EOS (end-of-sequence) tokens mark sequence boundaries in language models. **BOS purpose**: Signals sequence start, provides initial context token, allows model to begin coherent generation. Not all models use explicit BOS. **EOS purpose**: Signals completion, model learns to generate when done, critical for knowing when to stop inference. **Training**: Model sees EOS at end of training examples, learns association with completion. BOS at start provides consistent starting point. **Inference behavior**: Generate until EOS produced, then stop. Alternatively, stop at maximum length if EOS not generated. **Model variations**: Some use single token for both (like newlines in early GPT), others have distinct tokens. **Chat models**: May use turn-based end tokens instead of single EOS. **Implementation**: Check tokenizer documentation for specific token IDs and usage patterns. **Common issues**: Model not generating EOS (loops forever), generating EOS too early (truncated outputs). **Sampling interaction**: Temperature and sampling affect when EOS is chosen. May need to tune stopping criteria.