levenshtein transformer

**Levenshtein Transformer** is a **text generation model that generates and edits sequences using three edit operations: insertion, deletion, and replacement** — inspired by the Levenshtein edit distance, the model iteratively transforms an initial (possibly empty) sequence into the target through a series of learned edit steps. **Levenshtein Transformer Operations** - **Token Deletion**: Predict which tokens to delete — a binary classification at each position. - **Placeholder Insertion**: Predict where to insert new tokens — add placeholder positions for new tokens. - **Token Prediction**: Fill in the placeholder positions with actual tokens — predict the inserted tokens. - **Iteration**: Repeat deletion → insertion → prediction until convergence or a fixed number of steps. **Why It Matters** - **Edit-Based**: Natural for iterative refinement — the model can fix specific errors without regenerating the entire sequence. - **Adaptive Length**: Unlike fixed-length NAT, the Levenshtein Transformer can dynamically adjust output length through insertions and deletions. - **Flexible Decoding**: Can start from any initial sequence — including a rough draft, copied source, or empty sequence. **Levenshtein Transformer** is **text generation as editing** — building and refining sequences through learned insertion, deletion, and replacement operations.