enhanced mask decoder

**Enhanced Mask Decoder (EMD)** is a **component of DeBERTa that incorporates absolute position information in the final decoding layer** — compensating for the fact that disentangled attention uses only relative positions, which is insufficient for tasks like masked language modeling. **How Does EMD Work?** - **Problem**: Relative position alone cannot distinguish "A new [MASK] opened" → "store" vs "A new store [MASK]" → "opened". Absolute position matters. - **Solution**: Add absolute position embeddings only in the final decoder layer before the MLM prediction head. - **Minimal Disruption**: Most layers use relative position (better generalization). Only the decoder uses absolute position (for disambiguation). **Why It Matters** - **Position Disambiguation**: Absolute position is necessary for predicting masked tokens correctly in certain contexts. - **Best of Both**: Combines relative position (better generalization) with absolute position (necessary disambiguation). - **DeBERTa Architecture**: EMD is the third key innovation of DeBERTa alongside disentangled attention and virtual adversarial training. **EMD** is **the final position anchor** — adding absolute position information at the last moment so the model knows exactly where each prediction should go.