Home Knowledge Base BOS purpose

BOS (beginning-of-sequence) and EOS (end-of-sequence) tokens mark sequence boundaries in language models. BOS purpose: Signals sequence start, provides initial context token, allows model to begin coherent generation. Not all models use explicit BOS. EOS purpose: Signals completion, model learns to generate when done, critical for knowing when to stop inference. Training: Model sees EOS at end of training examples, learns association with completion. BOS at start provides consistent starting point. Inference behavior: Generate until EOS produced, then stop. Alternatively, stop at maximum length if EOS not generated. Model variations: Some use single token for both (like newlines in early GPT), others have distinct tokens. Chat models: May use turn-based end tokens instead of single EOS. Implementation: Check tokenizer documentation for specific token IDs and usage patterns. Common issues: Model not generating EOS (loops forever), generating EOS too early (truncated outputs). Sampling interaction: Temperature and sampling affect when EOS is chosen. May need to tune stopping criteria.

bos / eos tokensnlp

Explore 500+ Semiconductor & AI Topics

From EUV lithography to CUDA optimization — search the full knowledge base or chat with our AI assistant.