discourse marker prediction

**Discourse Marker Prediction** is a **self-supervised pre-training task where the model must predict missing or masked discourse markers (e.g., "however", "therefore", "because")** — forcing the model to understand the logical relationship between text segments (contrast, causality, elaboration) rather than just keyword co-occurrence. **Mechanism** - **Selection**: Identify discourse markers in the text using a predefined list. - **Masking**: Hide these markers. - **Task**: The model predicts the correct marker from a set of candidates. - **Example**: "He studied hard [MASK] he failed the test." -> Predict "yet" or "but". **Why It Matters** - **Coherence**: Discourse markers are the glue of logic — predicting them requires understanding the *argument* structure. - **NLI**: Improves performance on Natural Language Inference tasks (Entailment/Contradiction). - **Discrimination**: Helps models distinguish between correlation ("and") and causation ("because"). **Discourse Marker Prediction** is **logic gap-filling** — training the model to understand the logical flow of text by predicting the connecting words that define relationships.