Semantic Parsing and Meaning Representations › Limitations and Impact

The logical-form examples map language to logic, but still require a choice of representation. A practical difficulty is choosing a representation that captures what the task needs without capturing everything the sentence could mean. A parser targeting one fully scoped first-order sentence commits to a reading; an application can instead retain alternatives or underspecification. AMR can represent predicate-argument structure and reentrancy, but it deliberately abstracts away from scope and word order. SQL offers executable relational computations whose behavior depends on dialects, NULL semantics, and schema design. The representations discussed here can abstract away from linguistic distinctions. Whether a discarded distinction matters depends on the task and domain; a representation can preserve all distinctions stipulated by a restricted task.

A useful habit is to ask, for any representation, what it cannot say. If a task requires distinguishing inputs that map to the same object, no downstream system using only that object can answer correctly on all such inputs. If correct and incorrect alternatives map to the same object, a check using only that representation cannot tell them apart.

Evaluation is another limitation. Single-reference matching can reject valid alternatives; the outcome depends on the benchmark's normalization and equivalence rules. Execution accuracy is closer to the task, but a wrong query can return the right answer on a particular database. Smatch measures graph overlap, not truth conditions. Benchmark contamination, schema drift, and dialect differences can make a model look better or worse than it is. A semantic parser that scores well on Spider may still fail on your production schema because your column names, value formats, and joins are different. The lesson is to evaluate at multiple levels: validity, denotation, robustness to paraphrase, and behavior on unseen schemas. The evaluation ideas from Part LIII: Evaluation Fundamentals apply directly here.

It is also useful to evaluate the parser as a component, not just as a standalone model. A parser that is 90 percent accurate on individual queries may be much less accurate on a workflow that requires several queries in sequence.

Semantic parsing gives language models a way to act through structured interfaces whose outputs can be checked. Applications include databases, calendars, robots, spreadsheets, and APIs. It also connects natural language with program synthesis. When a model generates a SQL query, a function call, or a small program from a request, it is doing a kind of semantic parsing. Constrained generation can enforce a chosen output language; task correctness still needs semantic checks. A deployed interface can combine those checks with execution feedback and human review where the application calls for it. In the next chapters, we will see how these ideas connect to coreference, discourse, and dialogue, where context beyond a single sentence can matter.

The broader lesson is that meaning representations are interfaces. They are where linguistic meaning meets software behavior, and their design constrains which distinctions can pass from linguistic interpretation to downstream software.