winograd schema challenge

**Winograd Schema Challenge (WSC)** is a **commonsense reasoning benchmark consisting of pairs of sentences that differ by only one or two words, containing an ambiguous pronoun whose resolution resorts to world knowledge** — designed to be easy for humans but hard for machines. **Example** - **Sentence A**: "The trophy doesn't fit into the suitcase because **it** is too **large**." (It = Trophy). - **Sentence B**: "The trophy doesn't fit into the suitcase because **it** is too **small**." (It = Suitcase). - **Logic**: You must know physics (large things don't fit in small things) to resolve syntax. **Why It Matters** - **Turing Test Alternative**: Proposed by Hector Levesque as a better test of intelligence than the Turing Test. - **Evaluation**: Standard benchmark for LLMs. GPT-4 scores >90%, effectively "solving" it, though smaller models struggle. - **Selectional Restrictions**: Tests if models learn the physical/semantic constraints of verbs and adjectives. **Winograd Schema Challenge** is **the "It" test** — a benchmark testing if AI has enough commonsense physics to resolve ambiguous pronouns.