Semantic Parsing and Meaning Representations › Executable Meaning Representations

SQL is one example of an executable meaning representation. The broader idea is to map language to a formal object that can be run, checked, or interpreted. Execution lets a system compute a result when evaluation terminates and compare that result with an expected outcome. Non-executable representations can also support tasks such as graph matching and information extraction; a mathematical denotation does not itself require an effective executor.

Executable representations include:

• Logical formulas evaluated against a knowledge base or model.
• SQL queries executed against a relational database.
• Regular expressions matched against strings.
• Date and time expressions parsed into calendar objects.
• API calls such as set_alarm(time="07:00").
• Programs or DSL expressions that compute a value or perform an action.
• Planning operators that change the state of a robot or agent.

We can describe the semantics of an executable representation denotationally. We write $[\![m]\!]_{\mathcal{M}}$ for the denotation of representation $m$ in context $\mathcal{M}$. For a logical formula, the denotation is a truth value. For a SQL query, the denotation is a result table. For a program, the context can include its input and initial state; its denotation may describe a resulting value and state, with the chosen semantic model also accounting for failure or nontermination.

Denotational semantics is useful for learning because it lets us compare meanings without comparing strings. Two representations with the same denotation are observationally equivalent in that context for a task that observes only that result. Equivalence across changing databases requires agreement on every relevant instance and observation.

This gives us a powerful training signal. Instead of supervising the parser with gold logical forms, we can supervise it with denotations. The parser produces a representation, the representation is executed, and the result is compared to the correct answer. This is weak supervision. It is attractive because denotations can be cheaper to obtain than gold logical forms. Learning this way can be harder: the latent representation space may be large, and different representations can produce the same denotation.

A parser trained this way may learn a representation that happens to work on the training data but does not generalize. Credit assignment can be difficult because a final-answer mismatch need not identify which part of the representation caused the error.

Executable representations also enable verification and feedback. If a generated SQL query fails to execute, the error message can be used to repair it. If a generated program returns the wrong value, the difference can be used as a reward signal. This is one reason semantic parsing connects naturally to tool use and code generation, which we will explore in Part XLVIII: Tool Use and Agents and Part XLVII: Code Generation.

The feedback loop can be as simple as a retry with the error message prepended, or as complex as a search over candidate repairs. Either way, execution provides a direct signal for repair and learning.