global context block

**Global Context (GC) Block** is a **simplified and efficient version of the Non-Local block** — observing that Non-Local attention maps are nearly identical for different query positions, and replacing the per-query computation with a single global context vector shared across all positions. **How Does the GC Block Work?** - **Global Context**: $c = sum_j frac{exp(W_k x_j)}{sum_m exp(W_k x_m)} cdot x_j$ (attention-weighted global average). - **Transform**: $c' = ext{LayerNorm}(W_2 cdot ext{ReLU}(W_1 cdot c))$ (bottleneck transform like SE). - **Broadcast**: Add $c'$ to every spatial position: $y_i = x_i + c'$. - **Paper**: Cao et al. (2019). **Why It Matters** - **Efficiency**: One global context vector vs. N×N attention matrix -> dramatically cheaper than Non-Local. - **Same Quality**: Achieves similar or better results than Non-Local blocks at a fraction of the cost. - **Insight**: Revealed that query-independent attention is sufficient — you don't need per-pixel attention. **GC Block** is **Non-Local attention simplified** — the insight that one shared global context works as well as expensive per-position attention.