Span Boundary Objective (SBO) is a pre-training objective introduced in SpanBERT where the model must predict a masked span using ONLY the tokens at the span's boundaries — ensuring that the representation of boundary tokens captures the semantics of the entire contents between them.

Mechanism

• Task: For a masked span $(x_s, dots, x_e)$, predict each token $x_i$ in the span.
• Input: The tokens at the boundaries $x_{s-1}$ and $x_{e+1}$ PLUS the position embedding of the target $x_i$.
• Formula: $P(x_i) = f(x_{s-1}, x_{e+1}, pos_i)$.
• Constraint: The model CANNOT use the other masked tokens inside the span for context — only the edges.

Why It Matters

• Span Representations: Forces the boundary tokens to summarize the span content.
• Extractive Tasks: Extractive QA and Span Selection rely on start/end pointers — SBO directly optimizes these boundary representations.
• Performance: SpanBERT with SBO set SOTA on SQuAD and other span-based benchmarks.

Span Boundary Objective is judging a book by its covers — forcing the model to reconstruct a phrase using only the words immediately before and after it.