tokenization security
**Tokenization security** in the context of AI systems refers to protecting the **token-level interface** of language models from manipulation, injection, and exploitation. Since LLMs process text as sequences of tokens, understanding and securing this tokenization layer is critical for preventing attacks.
**Security Concerns**
- **Prompt Injection**: Attackers craft inputs with special token sequences that cause the model to **interpret injected instructions** as system-level commands, overriding intended behavior.
- **Token Boundary Exploitation**: Manipulating text so that the tokenizer splits it in unexpected ways, potentially bypassing content filters that operate on words rather than tokens.
- **Special Token Injection**: Inserting reserved special tokens (like `<|endoftext|>`, `[INST]`, ``) in user input that the model treats as control signals.
- **Unicode/Encoding Attacks**: Using invisible characters, zero-width joiners, homoglyphs, or unusual Unicode to create text that appears harmless but tokenizes into adversarial sequences.
**Protection Strategies**
- **Input Sanitization**: Strip or escape special tokens and control characters from user input before tokenization.
- **Token Allow/Deny Lists**: Block known adversarial token sequences or special tokens from appearing in user input.
- **Delimiter Enforcement**: Use robust delimiters between system prompts and user input that the model is trained to respect.
- **Post-Tokenization Validation**: Check the token sequence after tokenization for suspicious patterns before feeding it to the model.
- **Encoding Normalization**: Normalize Unicode to a canonical form (NFC/NFKC) before processing to prevent homoglyph and invisible character attacks.
**Challenges**
- **Tokenizer-Model Coupling**: Security filters must understand the specific tokenizer's behavior, as different tokenizers split text differently.
- **Evolving Attacks**: New token-level attacks are continuously discovered, requiring ongoing defense updates.
Tokenization security is a **critical layer** in the defense-in-depth strategy for LLM applications — it sits at the boundary between untrusted user input and the model's processing pipeline.