WordPiece is a subword tokenization algorithm used by BERT that builds vocabulary based on likelihood. Algorithm: Similar to BPE but selects merges by maximizing training data likelihood rather than raw frequency. Prefers merges that increase corpus probability. Training: Initialize with characters, iteratively merge pairs that maximize likelihood, continue until vocabulary size. Format: Continuation subwords marked with ## prefix (e.g., playing becomes play + ##ing). Advantages: Linguistically motivated merges, handles OOV through subword splits, efficient vocabulary usage. Used by: BERT, DistilBERT, ELECTRA, and their variants. Vocabulary size: BERT uses 30,522 tokens, including special tokens. Comparison to BPE: BPE uses frequency, WordPiece uses likelihood. Results often similar in practice. Special tokens: CLS, SEP, MASK, PAD, UNK for BERT-specific purposes. Implementation: HuggingFace tokenizers, original Google implementation. Legacy: While GPT-family uses BPE, BERT-family models continue using WordPiece, making it important for understanding encoder models.