executable semantic parsing

**Executable semantic parsing** is the NLP task of converting **natural language utterances into executable formal representations** — such as SQL queries, API calls, Python code, or logical forms — that can be directly run against a database, knowledge base, or programming environment to produce concrete answers or actions. **Why Executable Parsing?** - Traditional NLP often produces text answers — which may be vague, incomplete, or hallucinated. - **Executable parsing** produces structured, runnable code — the answer is computed by executing the generated program, ensuring precision and grounding in actual data. - The output is **verifiable**: you can check whether the generated code does what the user asked, and the execution result is deterministic. **Executable Parsing Pipeline** 1. **Natural Language Input**: User asks a question or gives a command in plain language. 2. **Semantic Parsing**: The model (LLM or specialized parser) converts the utterance into an executable representation. 3. **Execution**: The generated code or query is executed against the target system (database, API, interpreter). 4. **Result**: The execution output is returned to the user as the answer. **Target Representations** - **SQL**: For database queries — "How many customers are in New York?" → `SELECT COUNT(*) FROM customers WHERE state = 'NY'` - **SPARQL**: For knowledge graph queries — "Who directed Inception?" → `SELECT ?d WHERE { :Inception :director ?d }` - **Python/Code**: For calculations and data processing — "Plot sales by month" → Python code using pandas and matplotlib. - **API Calls**: For interacting with services — "Book a flight from NYC to London tomorrow" → structured API request. - **Lambda Calculus**: For compositional semantic representations — formal logical forms that can be evaluated. - **Robot Commands**: For embodied AI — "Pick up the red block" → structured action sequence. **Semantic Parsing with LLMs** - Modern LLMs have made executable semantic parsing much more accessible — they can generate SQL, Python, and API calls from natural language with high accuracy. - **In-context learning**: Few-shot examples of (question, code) pairs enable LLMs to parse new questions without fine-tuning. - **Schema/API awareness**: Providing the database schema or API documentation in the prompt helps the LLM generate syntactically and semantically correct code. **Challenges** - **Schema Grounding**: The parser must correctly map natural language terms to database columns, table names, and relationships. - **Compositional Generalization**: Handling complex, nested queries that combine multiple clauses — "Show customers who bought more than the average." - **Ambiguity**: Natural language is ambiguous — "top customers" could mean highest spending, most frequent, or most recent. - **Safety**: Executing generated code poses security risks — SQL injection, destructive operations, unauthorized access. **Evaluation** - **Execution Accuracy**: Does the generated code produce the correct answer when executed? (Preferred over exact match because multiple queries can produce the same result.) - **Benchmarks**: Spider (SQL), WikiTableQuestions, MTOP (API calls), GeoQuery. Executable semantic parsing is the **bridge between natural language and computation** — it transforms human intent into precise, executable actions, making databases, APIs, and code accessible to non-programmers.